# developer.siemens.com > Siemens Xcelerator Developer Portal Siemens Xcelerator Developer Portal documentation # Building X Openness documentation # Contact [Visit Building X on siemens.com](https://www.siemens.com/en-us/products/building-x/apis/) or [Contact our experts](https://www.siemens.com/en-us/products/building-x/contact/) to unleash the power of (your) building data with our use-case optimized Building X Openness APIs. Experience seamless integration and tap into valuable insights via Building X Openness. Optimize operations, drive efficiency, and unlock new opportunities. Start your integration journey today to harness the true potential of smart buildings. Together, let's transform the way we leverage data and shape the future of your building with Building X Openness. - [Overview](overview.html) - APIs & Services - Developer's Guide - [Getting Started](dev-guide/gettingstarted.html) - [General Information](dev-guide/index.html) - [Authentication](dev-guide/howto.html) - [Try Out APIs](dev-guide/try-out.html) - Accounts API - [Overview](api/accounts-api/overview.html) - [Quick Start](api/accounts-api/quickstart.html) - [API Reference](api/accounts-api/api-reference.html) - [Changelog](api/accounts-api/changelog.html) - Building Structure API - [Overview](api/building-structure/overview.html) - [Quick Start](api/building-structure/quickstart.html) - [API Reference](api/building-structure/api-reference.html) - [Changelog](api/building-structure/changelog.html) - Building Geometry API - [Overview](api/geometry-api/overview.html) - [Quick Start](api/geometry-api/quickstart.html) - [API Reference](api/geometry-api/api-reference.html) - [Changelog](api/geometry-api/changelog.html) - Building Operations API - [Overview](api/building-operations/overview.html) - [Quick Start](api/building-operations/quickstart.html) - [API Reference](api/building-operations/api-reference.html) - [Changelog](api/building-operations/changelog.html) - Point Value Ingest API - [Overview](api/point-value-ingest-api/overview.html) - [Quick Start](api/point-value-ingest-api/quickstart.html) - [API Reference](api/point-value-ingest-api/api-reference.html) - [Changelog](api/point-value-ingest-api/changelog.html) - Streaming API - [Overview](api/streaming-api/overview.html) - [Quick Start](api/streaming-api/quickstart.html) - [API Reference](api/streaming-api/api-reference.html) - [Changelog](api/streaming-api/changelog.html) - Energy API - [Overview](api/energy-api/overview.html) - [Quick Start](api/energy-api/quickstart.html) - [API Reference](api/energy-api/api-reference.html) - [Changelog](api/energy-api/changelog.html) - Sustainability API - [Overview](api/sustainability-api/overview.html) - [Quick Start](api/sustainability-api/quickstart.html) - [API Reference](api/sustainability-api/api-reference.html) - [Changelog](api/sustainability-api/changelog.html) - Security Identities and Privileges API - [Overview](api/security-api/piam-v1/overview.html) - [Quick Start](api/security-api/piam-v1/quickstart.html) - [API Reference](api/security-api/piam-v1/api-reference.html) - [Changelog](api/security-api/piam-v1/changelog.html) - Activities API - [Overview](api/activities-v1/overview.html) - [Quick Start](api/activities-v1/quickstart.html) - [API Reference](api/activities-v1/api-reference.html) - [Changelog](api/activities-v1/changelog.html) - Security Workflow API - [Overview](api/security-api/security-workflow-v1/overview.html) - [Quick Start](api/security-api/security-workflow-v1/quickstart.html) - [API Reference](api/security-api/security-workflow-v1/api-reference.html) - [Changelog](api/security-api/security-workflow-v1/changelog.html) - Security Intrusion API - [Overview](api/security-api/security-intrusion-v1/overview.html) - [Quick Start](api/security-api/security-intrusion-v1/quickstart.html) - [API Reference](api/security-api/security-intrusion-v1/api-reference.html) - [Changelog](api/security-api/security-intrusion-v1/changelog.html) - Security Monitoring API - [Overview](api/security-api/security-monitoring-v1/overview.html) - [Quick Start](api/security-api/security-monitoring-v1/quickstart.html) - [API Reference](api/security-api/security-monitoring-v1/api-reference.html) - [Changelog](api/security-api/security-monitoring-v1/changelog.html) - Visitor Manager API - [Overview](api/visitormanager-api/overview.html) - [Quick Start](api/visitormanager-api/quickstart.html) - [API Reference](api/visitormanager-api/api-reference.html) - [Changelog](api/visitormanager-api/changelog.html) - Fire API - [Overview](api/fire/overview.html) - [Quick Start](api/fire/quickstart.html) - [API Reference](api/fire/api-reference.html) - [Changelog](api/fire/changelog.html) - Lifecycle Twin API - [Overview](api/lifecycletwin-api/overview.html) - [Quick Start](api/lifecycletwin-api/quickstart.html) - [API Reference](api/lifecycletwin-api/api-reference.html) - [Changelog](api/lifecycletwin-api/changelog.html) - Developer Guide - General - [Assets](api/lifecycletwin-api/guide/assets.html) - [Querying Data](api/lifecycletwin-api/guide/querying.html) - [Create](api/lifecycletwin-api/guide/create.html) - [Update](api/lifecycletwin-api/guide/update.html) - [Delete](api/lifecycletwin-api/guide/delete.html) - [Authentication](api/lifecycletwin-api/guide/authentication.html) - [Tasks](api/lifecycletwin-api/guide/tasks.html) - Documents - [Get/Find a Document](api/lifecycletwin-api/examples/documents/find-document.html) - [Create a Document](api/lifecycletwin-api/examples/documents/create-document.html) - [Create a Folder](api/lifecycletwin-api/examples/documents/create-folder.html) - [Modify a Document](api/lifecycletwin-api/examples/documents/modify-document.html) - [Upload New Version](api/lifecycletwin-api/examples/documents/document-new-version.html) - [Download a Document File](api/lifecycletwin-api/examples/documents/download-document.html) - [Assemblies](api/lifecycletwin-api/examples/assemblies/assemblies.html) - Configuration - [Custom Fields](api/lifecycletwin-api/examples/customfields/custom-fields.html) - Custom Tabs - [Manage Tabs](api/lifecycletwin-api/examples/custom-tabs/manage-tabs.html) - [Manage Tab Fields](api/lifecycletwin-api/examples/custom-tabs/manage-tab-fields.html) - [Manage Values](api/lifecycletwin-api/examples/custom-tabs/manage-tab-values.html) - Animations - [Animations](api/lifecycletwin-api/examples/animations/model-animations.html) - [Definition](api/lifecycletwin-api/examples/animations/example-of-animation-definition.html) - RDS and Data Points - [RDS](api/lifecycletwin-api/examples/rds/rds.html) - [Data Points (RDS Variables)](api/lifecycletwin-api/examples/rds/data-points.html) - SDKs - [Security Mobile App SDK](sdk/security/security-mobile-app-sdk.html) - [Security Manager Access PACS SDK](sdk/security/security-manager-access-pacs-sdk.html) - Resources - [Mendix Connector](resources/mendix/overview.html) - Power BI Templates - Operations and Energy - [Overview](resources/power-bi-templates/power-bi-operations-dataset-overview.html) - [Tables](resources/power-bi-templates/power-bi-operations-dataset-tables.html) - [Contact](contact.html) # Enrich Building X With the Innovations You Create The power of Building X is its ability to pull all the data from your different systems, put it into a single data pool, and then let you easily connect to it. This new level of connectivity opens up a world of possibilities for making buildings smarter. ## Welcome to the Building X Ecosystem When you connect to the cloud, you're connecting to the future of smart buildings. Building X makes it easy, powerful, and cost-effective. At Building X, Openness is a paradigm that thrives on collaborative innovation by providing open protocols and seamless integration through robust, use-case optimized, subscription-based APIs. [Try out](./dev-guide/gettingstarted.html "Try out APIs") [Contact us](contact.html "Contact us") ## Open API Portfolio Unlock the business value of data with a comprehensive API portfolio and cloud data exchange. Access building data in the cloud using familiar technologies, such as RESTful APIs and applied API standards, to build your own building applications as needed. It's your gateway to maximizing your building's potential and creating smart buildings that redefine performance and sustainability solutions. ## Third-Party Apps Choose the apps that help you perform your daily tasks and deliver tangible business results. Add different applications at different times as your smart building systems grow or change. For application developers, integration with Building X is easy and affordable, making our APIs suitable for new smart building applications. ## A Platform for Innovation and Collaboration With its openness and ability to connect users to unified building data, Building X unleashes the power of collaboration to create a dynamic, market-leading smart building ecosystem. - Integration with Building X is simple and affordable, allowing applications to be created and deployed more cost-effectively. - Building owners benefit from a wider range of available applications at a lower total cost. - Service providers can leverage data and applications to create new, smarter offerings for their customers, powered by Building X. - Through its reach and accessibility, the Building X ecosystem continues to drive innovation and collaboration, accelerating the future of intelligent buildings. # Changelog The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). ### [1.1.0] - 2026-04-28 #### IMPORTANT CHANGES - **API Endpoint Update:** New domain and API path for the Account API. - **Old Endpoint:** `https://api.bpcloud.siemens.com/accounts/*` - **New Endpoint:** `https://eu.buildingx.siemens.com/api/openness/accounts/*` - **Action Required:** Please update your integrations to use the new endpoint. - **Grace Period:** The old domain will continue to be supported for a grace period. A separate notification will be sent to all customers regarding the deprecation date ### [1.0.2] - 2025-03-18 #### Added - Added support x-api-permission ### [1.0.1] - 2024-11-12 #### Fixed - Updated changelog. ### [1.0.0] - 2022-10-15 #### Added - Introducing the Building X - Accounts API, giving machine users information which partitions it can access # Accounts API The Accounts API enables you, or rather the calling entity, the machine user to discover in which partitions it has access. The Accounts API is organized around Customers, Partitions, UserGroups and Roles. ## Concepts & Glossary | Term | Description | | --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Customer | A Customer represents a company which owns resources like `Machine Users` and `Partitions`. | | Partition | A Partition represents a logical segmentation of a Customer. It can be used to group a number of Buildings together. | | Roles | A Role is a set of permissions representing which actions a machine user with a given Role can perform in the system. | | UserGroup | User groups allow users to organize Partitions and user access in a fine-grained way. UserGroups can be used to group machine users to give them access to one or multiple Partitions by acting in a certain Role. | # Quick Start Getting started with using Accounts APIs involves the following steps: 1. Create an account and a machine user. 1. Create a JSON Web Token (JWT) by using the machine user credentials. 1. Make API requests using the JWT. Note In the following examples we are: 1. making use of a Linux/MacOS shell in which environmental variables are set using the `export`-command. In other environments it may be different, e.g. Windows uses the `set`-command instead. 1. using the `curl` as a client. But the API can be used in any programming language with an HTTP Client, e.g. Go, Python, NodeJS, Javascript and Java. ## Create an account and a machine user The [Getting Started](../../dev-guide/gettingstarted.html) page documents the required steps to get a hold of the `clientId`, `clientSecret`. ## Create a token Use the values described in the [Authorization](../../dev-guide/howto.html)- section to construct the Create Token request. ### Example request ```bash export CLIENT_ID= export CLIENT_SECRET= curl https://siemens-bt-015.eu.auth0.com/oauth/token \ -H 'content-type: application/json' \ -d "{ \"client_id\":\"$CLIENT_ID\", \"client_secret\":\"$CLIENT_SECRET\", \"audience\":\"https://horizon.siemens.com\", \"grant_type\":\"client_credentials\" }" ``` To run this example yourself, set the `CLIENT_ID` and `CLIENT_SECRET` first. ### Example response ```json { "access_token": "eyJ0eXAiOiUSJ9.eyJpc3MiOiJdGlhbHMifQ.MJpcxLfyOt", "token_type": "Bearer", "expires_in": 86400 } ``` The token, or JWT (JSON Web Token), is the value of the `access_token`-property in the response. You can now use it by passing it in the `Authorization`-header of any subsequent API requests. The `expires_in`-property represents the number of seconds your token is valid, usually, the value corresponds to 24 hours. When this time has elapsed you will need to create a new token. Store the token in your `TOKEN`-environmental variable ```bash export TOKEN= ``` Now you have all you need to start using the API. ## Make API requests This guide will take you through the steps you need to: 1. Find out which partitions you can access 1. Getting additional information about Partitions and Customers ### List your UserGroups The first step to perform is to list the UserGroups you have access to. This you can do by performing the `Get the usergroups of the caller` operation i.e. `GET me/usergroups`. ```bash curl -H "Authorization: Bearer $TOKEN" \ "https://eu.buildingx.siemens.com/api/openness/accounts/me/usergroups" ``` The response contains all information about all UserGroups you (the machine user) has access to. If you have a large number of UserGroups you may need to retrieve multiple pages, see [Pagination](../../dev-guide/index.html#pagination). The most important information given for each UserGroup is the following: | Property | Description | | ---------------------------- | ------------------------------------------------------ | | `relationships.authorizedAs` | A list of Roles that the UserGroup has | | `relationships.hasAccessTo` | A list of Partitions that the UserGroups has access to | | `relationships.ownedBy` | A reference to the owning Customer | You have access to each partition mentioned above and are now able to use any other API e.g. `Building Operations API` with these partitions. To learn how to get then names of each Partition and Company please proceed to the next section. ### List Partitions Before you continue, store one of the Customer IDs for which you have access in an environmental variable, you can get the using the `Get the usergroups of the caller` operation as described above. ```bash export CUSTOMER=8db4216d-61c5-4e79-8558-164aa179bfe9 ``` To get more information about each of your Partitions you can use the `List Partitions for customer` operation. Below we specify the `include`-parameter to include the Company name of each Partition in the `included` section of the response. ```bash curl -H "Authorization: Bearer $TOKEN" \ "https://eu.buildingx.siemens.com/api/openness/accounts/customers/$CUSTOMER/partitions?include=ownedByCustomer.name" ``` You can repeat this operation for each of the Customers you have access to. Note For more details on deprecation policies and common API features such as paging, filtering and errors, refer to the [Developer's Guide](../../dev-guide/gettingstarted.html). ## Changelog The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). ### [1.2.0] - 2026-05-06 #### IMPORTANT CHANGES - **API Endpoint Update:** New domain and API path for the Security Activities API. - **Old Endpoint:** `https://api.bpcloud.siemens.com/sec-activities-v1/*` - **New Endpoint:** `https://eu.buildingx.siemens.com/api/openness/sec-activities-v1/*` - **Action Required:** Please update your integrations to use the new endpoint. - **Grace Period:** The old domain will continue to be supported for a grace period. A separate notification will be sent to all customers regarding the deprecation date ### [1.1.5] - 2026-01-20 #### Fix - Fixing issue with huge payloads. ### [1.1.4] - 2025-03-05 #### Update - Rename from Security Activities API to Activities API. ### [1.1.3] - 2024-11-12 #### Added - Added linting. ### [1.1.2] - 2024-08-08 #### Added - Aligned the documentation and corrected spelling errors ### [1.0.0] - 2023-12-20 #### Added - Security Activities API released - Added API endpoints for - Activities - To fetch all the activities # Activities API Activities API enables user to retrieve events, alarms and user operation produced from access system and security manager services. Customers can use the API to integrate the Security Manager functions & data into their own workflows & applications for reporting and investigation purpose. ## Concepts & Glossary | Term | Description | | -------- | -------------------------------------------------------------------------------------------------------------------------------------- | | Activity | Is an audit trail for Security Manager. Lists all events, alarms from access system and user operation from security manager services. | # Quick Start Getting started with using Activities APIs involves the following steps: 1. Create an account and a machine user. 1. Create a JSON Web Token (JWT) by using the machine user credentials. 1. Make API requests using the JWT. Note In the following examples we are: 1. Making use of a Linux/MacOS shell in which environmental variables are set using the `export` command. In other environments it may be different, e.g. Windows uses the `set` command instead. 1. Using the `curl` as a client. But the API can be used in any programming language with an HTTP Client, e.g. Go, Python, NodeJS, JavaScript and Java. ## Create an account and a machine user The [Getting Started](../../dev-guide/gettingstarted.html) page documents the required steps to get a hold of the `clientId`, `clientSecret` and `partitionId`. ## Create a token Use the values described in the [Authorization](../../dev-guide/howto.html) section to construct the Create Token request. ### Example request ```bash export CLIENT_ID= export CLIENT_SECRET= curl https://siemens-bt-015.eu.auth0.com/oauth/token \ -H 'content-type: application/json' \ -d "{ \"client_id\":\"$CLIENT_ID\", \"client_secret\":\"$CLIENT_SECRET\", \"audience\":\"https://horizon.siemens.com\", \"grant_type\":\"client_credentials\" }" ``` To run this example yourself, set the `CLIENT_ID` and `CLIENT_SECRET` first. ### Example response ```json { "access_token": "eyJ0eXAiOiUSJ9.eyJpc3MiOiJdGlhbHMifQ.MJpcxLfyOt", "token_type": "Bearer", "expires_in": 86400 } ``` The token, or JWT (JSON Web Token), is the value of the `access_token` property in the response. You can now use it by passing it in the `Authorization` header of any subsequent API requests. The `expires_in` property represents the number of seconds your token is valid, usually, the value corresponds to 24 hours. When this time has elapsed, you will need to create a new token. Now you have all you need to start using the API. As a last step of preparation set the token and `partitionId` as environmental variables. ```bash export PARTITION= export TOKEN= ``` ## Make API requests This guide will take you through the steps you need to: 1. Consume activities ### List Activities To fetch the user performed activities based by `PARTITION`. `size` parameter is an optional field and based on this value the response is returned (Note: `size` should not exceed 1000). `cursor` represent the last activity `id` from which the next set of activities can be fetched along with the size parameter. In case all the activities returned on the current request, the value of `next` will be null. ```bash curl -H "Authorization: Bearer $TOKEN" \ "https://eu.buildingx.siemens.com/api/openness/sec-activities-v1/partitions/$PARTITION/activities?page[size]=2" ``` The response contains all details of individual activities. ```bash { "links": { "self": "/activities?page[size]=2", "next": "/activities?page[size]=2&page[cursor]=6478632ea30ccb58b630b273" }, "data": [ { "type": "Activity", "id": "6478632ea30ccb58b630b272", "attributes": { "title": "Identity deleted", "description": "Identity Performance Test - Security API - First Name Performance Test - Security API - Last Name was deleted", "eventType": "Operator", "severity": "Low", "category": "User operation", "subCategory": null, "created": "2023-06-01T09:21:50.356Z", "recorded": "2023-06-01T09:21:52.868Z", "subSystem": "Siveillance Access", "origin": "Identity Management" } }, { "type": "Activity", "id": "6478632ea30ccb58b630b273", "attributes": { "title": "Identity created", "description": "Identity Performance Test - Security API - First Name Performance Test - Security API - Last Name was created", "eventType": "Operator", "severity": "Low", "category": "User operation", "subCategory": null, "created": "2023-06-01T09:21:50.356Z", "recorded": "2023-06-01T09:21:50.758Z", "subSystem": "Siveillance Access", "origin": "Identity Management" } } ] } ``` Note For more details on deprecation policies and common API features such as paging, filtering and errors, refer to the [Developer's Guide](../../dev-guide/gettingstarted.html). # Changelog The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). ### [1.5.0] - 2026-05-06 #### IMPORTANT CHANGES - **API Endpoint Update:** New domain and API path for the Building Operation API. - **Old Endpoint:** `https://api.bpcloud.siemens.com/operations/*` - **New Endpoint:** `https://eu.buildingx.siemens.com/api/openness/operations/*` - **Action Required:** Please update your integrations to use the new endpoint. - **Grace Period:** The old domain will continue to be supported for a grace period. A separate notification will be sent to all customers regarding the deprecation date ### [1.4.0] - 2025-10-01 #### Added - Add Fault API endpoints: - `GET /faults` - Get all the faults that were active in a specified time range according to the given query parameters. - `GET /faults/{faultId}` - Get the specified Fault. - `PATCH /faults/{faultId}` - Update the state of Fault. - `GET /faults/{faultId}/occurrences` - Get all the occurrences for the given fault. - `GET /fault-statistics` - Get fault statistics for the given building. - Add WorkOrder API endpoints: - `GET /workorders` - Get the list of all workorders. - `POST /workorders` - Creates a new workorder (supports Alarm-based, Fault-based, and Generic types). - `GET /workorders/{workorderId}` - Get the workorder for the given workorderId. - `PATCH /workorders/{workorderId}` - Update the workorder for the given workorderId. - `GET /workorders/{workorderId}/assignments` - Get a list of assignments for a given workorder id. - `POST /workorders/{workorderId}/assignments` - Add assignment to workorder. - `PUT /workorders/{workorderId}/assignments` - Set Assignments for the Workorder. - Add Teams API endpoints: - `GET /teams` - Retrieves a list of all teams that match the specified filters. - `POST /teams` - Creates a new team for a customer with the specified name, description, domain, and assigned buildings. - `GET /teams/{teamId}` - Get the details of specific team. - `PATCH /teams/{teamId}` - Updates an existing team's name, description, domain, and buildings. - `GET /teams/{teamId}/members` - Retrieves a list of all members within a specific team. - `PATCH /teams/{teamId}/members` - Add a list of members within a specific team. ### [1.3.17] - 2024-11-12 #### Fixed - Updated changelog. ### [1.3.16] - 2024-08-29 #### Fixed - Fix code responses in specification. ### [1.3.15] - 2024-08-08 #### Added - Aligned the documentation and corrected spelling errors ### [1.3.5] - 2023-06-06 #### Added - 'Add point values' operation. Enabling import of point values. Point value ingestion is exposed for evaluation purposes until the new subscription is released by September 2023. - 'Create device' operation. Device creation is exposed for evaluation purposes until the new subscription is released by September 2023. ### [1.3.0] - 2023-03-20 #### Added - Adding Alarm Configuration management operations - 'List event transition history'-operation #### Updated - Deprecated 'Get Event', which is replaced with 'List event transition history'. ### [1.2.0] - 2022-07-07 #### Added - Added multiple new properties to `Point` model for a uniform access, independent of Edge Application: - description - isActive - unit - enum - minimum - maximum - precision - commandingSemantic - function - Added new properties to `PointValue` model for a uniform access, independent of Edge Application: - targetValue - qualityOfValue #### Updated - Removed the `pattern` definition and any uuid references as IDs may have other formats ### [1.0.7] - 2022-05-22 #### Fixed - Multiple minor documentation improvements, regarding `include`-params and device relationships ### [1.0.5] - 2022-04-21 #### Fixed - Removed `included` from response model of 'Get Single Device' as it is not supported ### [1.0.4] - 2022-03-22 #### Added - First version of 'Operations API' # Building Operations API The Building Operations API enables you to read data from data points or issue commands to actuator points. Commanding enables you to e.g. change set-point values on a thermostat or open or close a valve. The Building Operations API is organized around the `Device`-, `Point`- and `PointValue`- resources. A device can host several points capable of either sensing information from the building such as temperature or air quality, or capable of receiving and processing commands. The data that is measured by data points is called point values and you can specify the time interval for which to fetch data. ## Concepts & Glossary | Term | Description | | ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Device | A Device can be any electronic equipment with some computing ability that has a firmware or supports the installation of software. Examples are pumps, dampers, valves, sensors, detectors, limit switches, remote/local switches or automation devices. | | Point | Points represent the interface to data in a system. Points are classifying data and give a semantic context to it. Examples for a Point are a Room Control for Temperature with a temperature set point. | | PointValue | Object which represents a measurement or set value of a Point. | | Event | An Event represents a non-normal (non-quiescent) state in building automation systems and is caused by an abnormal situation (technical: like fault, alarm, detector exclusion, range violation but also human: manually creating an alarm/task, request/call). | | Location | A structure or part of a structure where a Device can be located | | Workorder | A Workorder represents a task or job that needs to be carried out within the building operations, such as maintenance, repair, or inspection activities. Workorders can be created, updated, retrieved, and assigned to specific teams or individuals. They provide a structured way to manage operational activities and ensure proper tracking of work progress. | | Fault | A Fault represents a detected abnormal condition or malfunction within a building system. Faults can be monitored, updated, and analyzed to understand their occurrences and overall impact on the building’s performance. | | Team | A Team represents a group of people responsible for performing tasks or managing specific areas of the building operations. Teams can be created, updated, retrieved, and managed to ensure collaboration and assignment of responsibilities. | A building typically contains multiple different `Devices`. The `Devices` host `Points` which can be of the following types: sensor, command or set-point. `Points` that are either sensors or set-points can have measured or aggregated time-series `PointValues` which can be filtered by time interval. ## Devices Some devices are connected directly to the Building X cloud and some make use of other devices to manage the connection. When a device relies on another device to connect to the cloud, it has a relationship called `hasGateway` defined, referring to the device it uses as a gateway to Building X. ## Points Any device may have points, regardless of whether the device is directly connected to the cloud or not. A building system can generate Faults that highlight abnormal conditions, which in turn lead to Workorders documenting the required activities, and these Workorders are assigned to Teams that ensure the issues are resolved effectively. ## Workorders Workorders are used to manage tasks and operational activities in the building environment. They enable a structured approach to handling different kinds of work, ensuring proper planning, execution, and tracking. A workorder can be created to document a maintenance activity, an inspection, or a repair. Once created, it can be updated with additional details such as its description, priority, or due date. Workorders can then be assigned to individuals or teams so that responsibilities are clear, and their progress can be tracked throughout the lifecycle of the task. They serve as a reliable tool for monitoring the status of ongoing activities, keeping a history of what has been completed, and ensuring that all work within the building operations is properly documented and followed through until completion. ## Faults Faults represent abnormal conditions or malfunctions in building systems and play an important role in monitoring operational health. They allow building operators to identify and analyze issues that may affect performance or safety. Faults can be retrieved to get an overview of all the problems detected in a partition or for a specific fault identifier. The state of a fault can be updated as its condition evolves, for instance when it is acknowledged or resolved. Each fault can also be examined in detail by looking at its occurrences, which provide insights into how frequently the issue arises and under what circumstances. Additionally, aggregated statistics about faults in a building can be accessed to better understand their overall impact, supporting both reactive and preventive maintenance strategies. In this way, the management of faults ensures that technical issues are not only detected but also documented, analyzed, and addressed effectively. ## Teams Teams represent groups of people that are responsible for carrying out tasks or managing specific aspects of building operations. They can be created for a customer with a defined name, description, and domain, and can be linked to certain locations or buildings. Once created, a team can be updated to reflect changes in its responsibilities or organizational context. Teams can also be retrieved to view details such as their members, scope, and associated locations. Members can be added to or managed within a team, ensuring that the right individuals are assigned to the right groups. This structure supports collaboration, accountability, and clear allocation of work across the building operations. Teams therefore serve as a foundation for organizing human resources, enabling efficient execution of workorders, and ensuring smooth communication and responsibility management within the building environment. ### System Attributes In Building X some point properties are completely managed by the platform. In the Building Operations API those are defined in the API specification and include `dataType` and `name`. In addition, there are other properties contained in the `systemAttributes`-object. The content of the `systemAttributes` is not marshaled by the Building Operations API and depends on the SW versions and configuration of devices and gateways connecting to Building X. As such, they may appear differently depending on the building and mentioned configurations. In addition, these properties may change outside of the API lifecycle, e.g., when updating firmware of the gateway device or software of the edge application. # Quick Start Getting started with using Building Operations APIs involves the following steps: 1. Create an account and a machine user. 1. Create a JSON Web Token (JWT) by using the machine user credentials. 1. Make API requests using the JWT. Note In the following examples we are: 1. making use of a Linux/MacOS shell in which environmental variables are set using the `export`-command. In other environments it may be different, e.g. Windows uses the `set`-command instead. 1. using the `curl` as a client. But the API can be used in any programming language with an HTTP Client, e.g. Go, Python, NodeJS, Javascript and Java. ## Create an account and a machine user The [Getting Started](../../dev-guide/gettingstarted.html) page documents the required steps to get a hold of the `clientId`, `clientSecret` and `partitionId`. ## Create a token Use the values described in the [Authorization](../../dev-guide/howto.html)- section to construct the Create Token request. ### Example request ```bash export CLIENT_ID= export CLIENT_SECRET= curl https://siemens-bt-015.eu.auth0.com/oauth/token \ -H 'content-type: application/json' \ -d "{ \"client_id\":\"$CLIENT_ID\", \"client_secret\":\"$CLIENT_SECRET\", \"audience\":\"https://horizon.siemens.com\", \"grant_type\":\"client_credentials\" }" ``` To run this example yourself, set the `CLIENT_ID` and `CLIENT_SECRET` first. ### Example response ```json { "access_token": "eyJ0eXAiOiUSJ9.eyJpc3MiOiJdGlhbHMifQ.MJpcxLfyOt", "token_type": "Bearer", "expires_in": 86400 } ``` The token, or JWT (JSON Web Token), is the value of the `access_token`-property in the response. You can now use it by passing it in the `Authorization`-header of any subsequent API requests. The `expires_in`-property represents the number of seconds your token is valid, usually, the value corresponds to 24 hours. When this time has elapsed you will need to create a new token. Now you have all you need to start using the API. As a last step of preparation set the token and `partitionId` as environmental variables. ```bash export PARTITION= export TOKEN= ``` ## Make API requests This guide will take you through the steps you need to perform to read values, e.g. temperature measurements, from a Point. We will assume you don't yet know the id of the Point, meaning that before you can get the temperature data for the Point you must discover the Point by listing Locations, Devices and Points. ### List Locations The first step to perform is to list the buildings in your partition. This you can do by performing the `List Locations` operation. Below we specify the `include` -parameter to include the address of each building in the `included` section of the response. ```bash curl -H "Authorization: Bearer $TOKEN" \ "https://eu.buildingx.siemens.com/api/openness/operations/partitions/$PARTITION/locations?filter[type]=Building&include=hasPostalAddress" ``` The response contains all buildings in your partition. If you have a large number of buildings you may need to retrieve multiple pages, see [Pagination](../../dev-guide/index.html#pagination). Select the `id` property of one of the locations in the response and set it in an environmental variable. E.g. if the `id` is `8db4216d-61c5-4e79-8558-164aa179bfe9` then set it using the following command: ```bash export LOCATION=8db4216d-61c5-4e79-8558-164aa179bfe9 ``` ### List Devices The next step is to list the devices available in your building. This is achieved by using the `List Devices` operation. In this example also the optional `include` -parameter is specified, it's useful to retrieve the name and connectivity status of the devices. ```bash curl -H "Authorization: Bearer $TOKEN" \ "https://eu.buildingx.siemens.com/api/openness/operations/partitions/$PARTITION/devices?filter[hasLocation.data.id]=$LOCATION&include=hasFeatures.DeviceInfo,hasFeatures.Connectivity" ``` The response contains a list of all devices with their location defined as the building you specified. To also get the devices that are potentially behind the first set of devices we must use the `List devices behind a gateway` operation. This must be done for each device retrieved above. Imagine we have retrieved one device with id `7d4293a7-5c61-47e2-b010-9ba913c016e4`, then we can get the devices behind it by performing the following operation: ```bash curl -H "Authorization: Bearer $TOKEN" \ "https://eu.buildingx.siemens.com/api/openness/operations/partitions/$PARTITION/devices/7d4293a7-5c61-47e2-b010-9ba913c016e4/devices&include=hasFeatures.DeviceInfo,hasFeatures.Connectivity" ``` Once you have performed the second request for each of the devices in the first set you have all the devices in your building. Any of these devices may host points. That means that to be certain to retrieve all points in your building you must perform the `List Points` operation, explained in the next section, for each of the devices. If your use case is well defined and you have a well-known hierarchy in your setup you can of course use that knowledge, e.g. by using the `modelName` or `DeviceInfo` to only query some devices for points. ### List Points for Device The `List Points` operation retrieve point for a certain device. By specifying the optional `field`-parameter, also the latest known value will be present in the response. Assuming the `DEVICE` environmental variable contains the id of the device you are interested in you can perform the following request: ```bash curl -H "Authorization: Bearer $TOKEN" \ "https://eu.buildingx.siemens.com/api/openness/operations/partitions/$PARTITION/devices/$DEVICE/points?field[Point]=pointValue" ``` The response contains a list of all points for the device. Select the `id` property and store in the `POINT`-variable. E.g. ```bash export POINT=a850ec9a-daf6-4630-9c22-af680e82b3a4 ``` ### List Values To get the time series data or PointValues you can use the `List Values` operation. To use this operation you will need to specify the query parameter `filter[timestamp][from]` with a valid date-time in the RFC3339 (UTC) format. The recommended value for trying is to specify the current time minus one hour. E.g. `2021-10-25T23:00:00.000Z` ```bash curl -H "Authorization: Bearer $TOKEN" \ "https://eu.buildingx.siemens.com/api/openness/operations/partitions/$PARTITION/points/$POINT/values?filter[timestamp][from]=2021-10-25T23:00:00.000Z" ``` In the response, you will find the latest values, e.g. temperature or air quality readings for the point. ### Commanding Points You can change the value of a point by using the `Update Point` operation. The following example will set the value of the point to `2`. ```bash curl -X PATCH "https://eu.buildingx.siemens.com/api/openness/operations/partitions/$PARTITION/points/$POINT" \ -H "Content-Type: application/vnd.api+json" \ -H "Authorization: Bearer $TOKEN" \ -d "{ \"data\": { \"type\": \"Point\", \"id\": \"$POINT\", \"attributes\": { \"pointValue\": { \"value\": \"2\" } } } }" ``` Note Even though the `Update Point` operation itself is successful, it does not mean that the point's state is reflected immediately. There can be a delay of up to a minute until the value is applied to the point. It's also possible that the command is discarded due to automation applications with higher priority already commanding the point. ### List Events To get current and past events and alarms for a device, the `List Events` operation can be used: ```bash curl -H "Authorization: Bearer $TOKEN" \ "https://eu.buildingx.siemens.com/api/openness/operations/partitions/$PARTITION/devices/$DEVICE/events" ``` ### Create Device To create a new device, the `Create Device` operation can be used: ```bash curl -X 'POST' \ "https://eu.buildingx.siemens.com/api/openness/operations/partitions/$PARTITION/devices" \ -H "Accept: application/vnd.api+json" \ -H "Authorization: Bearer $TOKEN" \ -H "Content-Type: application/vnd.api+json" \ -d "{ \"data\": { \"type\": \"Device\", \"attributes\": { \"modelName\": \"string\", \"serialNumber\": \"string\" } } }" ``` In the response, you will find the device is created for the given modelName & serialNumber. ### Add point values To add point values, the `Post values` operation can be used. Push 1 to 100 values for given point in a batch. Only one pointValue per millisecond will be stored. pointValues are dropped if there are timestamp conflicts, i.e. if a pointValue with the same timestamp already exists. Example with following pointValues: - 2022-07-11T11:11:30.333 - 2022-07-11T11:11:30.222 - 2022-07-11T11:11:30.333 - 2022-07-11T11:11:30.444 The first is accepted (it's the first for time "30.333"). The second is also accepted. The third is dropped because it's the same timestamp as the first. The fourth is accepted again. The indices of dropped pointValues are listed in the response (0-based), in this example there is one conflict with index "2". If there is no conflict status 204 is returned instead of 202. Note: Use query parameter `overwrite=true` to force overwriting of conflicting values. ```bash curl -X 'POST' \ "https://eu.buildingx.siemens.com/api/openness/operations/partitions/$PARTITION/points/$POINTID/values" \ -H "Accept: application/vnd.api+json" \ -H "Authorization: Bearer $TOKEN" \ -H "Content-Type: application/vnd.api+json" \ -d "{ \"data\": [ { \"type\": \"PointValue\", \"attributes\": { \"timestamp\": \"2000-05-11T10:32:54.964Z\", \"value\": \"12.66\" } } ] }" ``` In the response, you will find the values added for the pointId. Note For more details on deprecation policies and common API features such as paging, filtering and errors, refer to the [Developer's Guide](../../dev-guide/gettingstarted.html). # Changelog The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). ### [1.1.0] - 2026-04-28 #### IMPORTANT CHANGES - **API Endpoint Update:** New domain and API path for the Structure API. - **Old Endpoint:** `https://api.bpcloud.siemens.com/structure/*` - **New Endpoint:** `https://eu.buildingx.siemens.com/api/openness/structure/*` - **Action Required:** Please update your integrations to use the new endpoint. - **Grace Period:** The old domain will continue to be supported for a grace period. A separate notification will be sent to all customers regarding the deprecation date. ### [1.0.35] - 2025-09-12 - Enrich Point with additional attributes: unit, enum, systemAttributes ### [1.0.34] - 2025-02-04 - Fixed maestro version in order to realign to 1.0.31 - Fixed vulnerabilities ### [1.0.33] - 2024-11-12 #### Updated - Updated changelog ### [1.0.32] - 2024-11-12 #### Updated - Aligned the documentation and corrected spelling errors ### [1.0.30] - 2024-06-18 #### Updated - Fix minor issue with schema from 'Create Equipment' and 'Update Equipment' ### [1.0.28] - 2024-03-12 #### Updated - Updating spec from `Structure API` -> `Building Structure API`. ### [1.0.25] - 2023-09-29 #### Updated - Fixes to specification to reflect that client generated IDs are not supported when creating Point and Equipment. ### [1.0.24] - 2023-06-06 #### Added - 'Create Point' operation. Creation of points is exposed for evaluation purposes until the new subscription is released by September 2023. ### [1.0.23] - 2023-05-25 #### Updated - Fix typo of "writeable" - Fix inconsistency, "/has-points" should take an array as primary data ### [1.0.22] - 2023-05-15 #### Added - Point.source property ### [1.0.16] - 2022-12-15 #### Added - Equipment, PointGroups endpoints to enable management of building equipment topology ### [1.0.1] - 2022-09-28 #### Updated - Location ID schema ### [1.0.0] - 2022-09-27 #### Added - Structure API released # Building Structure API The Building Structure API enables you to create and manage a structural representation of your buildings in Building X. The API also helps you manage the location of equipment inside buildings and enables you to programmatically automate data setup. The Building Structure API is organized around elements of building structures, called Locations, e.g. Building, Floor and Room ## Concepts & Glossary | Term | Description | | -------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Location | A Location is the basic element describing the topological structure of a site or building, i.e. the common base class of a Campus, Building, Floor, Room etc. | | Campus | A Campus is a grouping of multiple Buildings and Outside spaces (directly or via CampusParts). Typically it is physically represented by a fence, vegetation or street. It could be a contiguous area like an enfenced airport or a looser aggregation like an government district (with foreign buildings between). | | CampusPart | In larger or disjoint campuses, there might be the need for a subsection, e.g. the 'science campus' or the 'west shore campus'. | | Building | A building is generally a structure built for a purpose that has a roof and walls and stands more or less permanently in one place. Typical examples are a house, factory, or shop; but if necessary also a cave, tower, tent or even a ship could be called a building. | | BuildingPart | In a larger or structured building, there might be the need for a subgrouping, e.g. the 'left wing' or the 'south part'. | | Outside | On a campus or even as part of a building, there can be outside areas, e.g. the lawns between buildings, the pavement to the end of the building lot, the carpark or delivery ramp, or a balcony or roof-top restaurant. | | Floor | A floor is a horizontal / level part of a building with a surface that could be used by people (for living, work, storage, recreation, etc), depending on context, they are also called storey/story, level or deck. Multiple floors can be stacked up vertically to form a multi-floor building. | | FloorArea | A floor-area is a subsection of an floor, which - unlike rooms - is not separated by walls to which entry is possible by a door or similar as to the floor itself. Normally, areas are large enough for many persons to move about or conduct the intended activity. | | MultifloorArea | A multifloor-area is a vertical crosssection of multiple floors, like a lecture room, cinema theatre, lobby that spreads the height of multiple floors and can be typically accessed on multiple levels. | | Room | A room is an indoor space enclosed within four walls (and a roof or ceiling) to which entry is possible by a door or similar, which connects it either to a passageway, to another room, or to the outdoors. Normally, rooms are large enough for several persons to enter and move about or conduct the intended activity. Typically, the height of a room is the full height of the floor it is part of. | | RoomSegment | Often, a floor is implicitly divided into a grid or into segments that later on might turn into rooms by adding walls. Such segments often contain technical preparations for possible rooms (e.g. pillars, grids, window arrangement, piping or wiring). | | Device | A Device can be any electronic equipment with some computing ability that has a firmware or supports the installation of software. Examples are pumps, dampers, valves, sensors, detectors, limit switches, remote/local switches or automation devices. | # Quick Start Getting started with using Building Structure APIs involves the following steps: 1. Create an account and a machine user. 1. Create a JSON Web Token (JWT) by using the machine user credentials. 1. Make API requests using the JWT. Note In the following examples we are: 1. making use of a Linux/MacOS shell in which environmental variables are set using the `export`-command. In other environments it may be different, e.g. Windows uses the `set`-command instead. 1. using the `curl` as a client. But the API can be used in any programming language with an HTTP Client, e.g. Go, Python, NodeJS, Javascript and Java. ## Create an account and a machine user The [Getting Started](../../dev-guide/gettingstarted.html) page documents the required steps to get a hold of the `clientId`, `clientSecret` and `partitionId`. ## Create a token Use the values described in the [Authorization](../../dev-guide/howto.html)- section to construct the Create Token request. ### Example request ```bash export CLIENT_ID= export CLIENT_SECRET= curl https://siemens-bt-015.eu.auth0.com/oauth/token \ -H 'content-type: application/json' \ -d "{ \"client_id\":\"$CLIENT_ID\", \"client_secret\":\"$CLIENT_SECRET\", \"audience\":\"https://horizon.siemens.com\", \"grant_type\":\"client_credentials\" }" ``` To run this example yourself, set the `CLIENT_ID` and `CLIENT_SECRET` first. ### Example response ```json { "access_token": "eyJ0eXAiOiUSJ9.eyJpc3MiOiJdGlhbHMifQ.MJpcxLfyOt", "token_type": "Bearer", "expires_in": 86400 } ``` The token, or JWT (JSON Web Token), is the value of the `access_token`-property in the response. You can now use it by passing it in the `Authorization`-header of any subsequent API requests. The `expires_in`-property represents the number of seconds your token is valid, usually, the value corresponds to 24 hours. When this time has elapsed you will need to create a new token. Now you have all you need to start using the API. As a last step of preparation set the token and `partitionId` as environmental variables. ```bash export PARTITION= export TOKEN= ``` ## Make API requests This guide will take you through the steps you need to: 1. Consume an existing Building, including all parts, possibly created using the Data Setup App 1. Creating a Building hierarchy ### List Buildings The first step to perform is to list the buildings in your partition. This you can do by performing the `List Locations` operation. Below we specify the `include` -parameter to include the address of each building in the `included` section of the response. ```bash curl -H "Authorization: Bearer $TOKEN" \ "https://eu.buildingx.siemens.com/api/openness/structure/partitions/$PARTITION/locations?filter[type]=Building&include=hasPostalAddress" ``` The response contains all buildings in your partition. If you have a large number of buildings you may need to retrieve multiple pages, see [Pagination](../../dev-guide/index.html#pagination). Select the `id` property of one of the locations in the response and set it in an environmental variable. E.g. if the `id` is `8db4216d-61c5-4e79-8558-164aa179bfe9` then set it using the following command: ```bash export LOCATION=8db4216d-61c5-4e79-8558-164aa179bfe9 ``` ### List Parts of a Building The next step is to list the parts of the building. This is achieved by using the `List Location Parts` operation. ```bash curl -H "Authorization: Bearer $TOKEN" \ "https://eu.buildingx.siemens.com/api/openness/structure/partitions/$PARTITION/locations/$LOCATION/relationships/has-parts" ``` The response contains a list of direct and transitive parts of the specified location with their location defined as the building you specified. I.e. if your building has Floors and those Floors have Rooms the response will contain all those Floors and Rooms. If you wish to perform some logic on the Building hierarchy or visualize a tree you need to assemble the tree yourself. It is straightforward and done by using the hierarchical relationships e.g. `isFloorOf` and `isRoomOf`. ### Creating a Location Hierarchy All locations have a parent, or in other words, they have an `isPartOf`-relationship to a higher level Location, except Campus and Building. Buildings are optionally related to Campuses or CampusParts. This means to create a hierarchy we must start with the top-level Location. To create a Building we also must provide a reference to and `Address`-resource. Optionally in can be provided using the `include`-feature of the API, we will use this in our example. Create a file named `newbuilding.json`. Paste in the following content below but please change the `id` of the Address to something unique, e.g. by creating a new UUID using [uuidgenerator.net](https://www.uuidgenerator.net/). Note that 'data/relationships/data/id' must match the 'included[0]/id' ```json { "data": { "type": "Building", "attributes": { "label": "My Place", "timeZone": "Europe/Zurich" }, "relationships": { "hasPostalAddress": { "data": { "id": "05085306-9c1d-4cee-b92a-04c07e4b1945", "type": "Address" } } } }, "included": [ { "id": "05085306-9c1d-4cee-b92a-04c07e4b1945", "type": "Address", "attributes": { "locality": "Springfield", "countryCode": "USA", "region": "Indiana", "postalCode": "10101", "street": "33 Park Way" } } ] } ``` You are now ready to create the Building by performing the `Create Location` operation: ```bash curl -H "Authorization: Bearer $TOKEN" \ -X POST -d @newbuilding.json -H "Content-Type: application/vnd.api+json" \ "https://eu.buildingx.siemens.com/api/openness/structure/partitions/$PARTITION/locations" ``` In the response, you will find the generated id for your Building. Now you can continue by creating floors and using the Building-ID. If the BuildingID was `79d8fd89-6c82-40b6-aa5a-9a570d3ae88a` the request body to create a floor would look like this: ```json { "data": { "type": "Floor", "attributes": { "label": "My Floor", "floorNumber": 1 }, "relationships": { "isFloorOf": { "data": { "id": "79d8fd89-6c82-40b6-aa5a-9a570d3ae88a", "type": "Building" } } } } } ``` ### Add points to point-groups To add points to point-groups, the `Post /point-groups/{point-groupId}/points` operation can be used. ```bash curl -X 'POST' \ "https://eu.buildingx.siemens.com/api/openness/structure/partitions/$PARTITION/point-groups/$POINTGROUPID/points" \ -H "Accept: application/vnd.api+json" \ -H "Authorization: Bearer $TOKEN" \ -H "Content-Type: application/vnd.api+json" \ -d "{ \"data\": { \"type\": \"Point\", \"attributes\": { \"name\": \"hvac 2134444\", \"description\": \"HVAC controller\", \"dataType\": \"number\", \"maximum\": 100, \"minimum\": 100, \"precision\": 0.1, \"commandingSemantic\": \"writeable\", \"function\": \"sensor\" } } }" ``` In the response, it will add points to the point-groups. Note For more details on deprecation policies and common API features such as paging, filtering and errors, refer to the [Developer's Guide](../../dev-guide/gettingstarted.html). # Changelog The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). ### [1.3.0] - 2026-05-5 #### IMPORTANT CHANGES - **API Endpoint Update:** New domain and API path for the Energy API. - **Old Endpoint:** `https://api.bpcloud.siemens.com/energy/v1/*` - **New Endpoint:** `https://eu.buildingx.siemens.com/api/openness/energy/v1/*` - **Action Required:** Please update your integrations to use the new endpoint. - **Grace Period:** The old domain will continue to be supported for a grace period. A separate notification will be sent to all customers regarding the deprecation date ### [1.2.6] - 2026-03-13 #### Added - Added support x-api-permission. ### [1.2.5] - 2024-11-12 #### Added - Added linting. ### [1.2.4] - 2024-08-08 #### Added - Aligned the documentation and corrected spelling errors ### [1.1.0] - 2024-01-12 #### Added - First version of `Energy API` ### [1.1.1] - 2024-01-22 #### Added - Spec update - requirements for deploy Energy API gateway ### [1.1.2] - 2024-01-23 #### Added - Spec update for related envs # Energy API The Energy API enables you to read energy consumption, emission, or cost data for a location or a meter in the granularity and for the period of interest. The API is available as an add-on in combination with an active Building X Energy Manager subscription. Building X Energy Manager customers with a `Navigator conversion` subscription cannot use the API. ## Concepts & Glossary | Term | Description | | ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Medium-consumption | A Medium-Consumption determines the consumption of some medium for some discipline (today `consumption groups`) of a location (e.g., building) or a meter equipment over a period. | | Consumption | Consumption provides the consumption of a location or measured by a Meter-Equipment that is derived from its meter readings. | | Costs | Consumption provides the consumption of a location or measured by a Meter-Equipment that is derived from its meter readings. | | Emissions | Emissions provide the emissions calculated based on defined emission conversion factors in Building X Energy Manager and the measured consumption. | *Usage of APIs is subject to an agreement between customer and Siemens. To the extent nothing specific is agreed, the Siemens terms and conditions agreed in the Siemens subscription manager shall apply.* # Quick Start Getting started with using Energy APIs involves the following steps: 1. Create an account and a machine user. 1. Create a JSON Web Token (JWT) by using the machine user credentials. 1. Make API requests using the JWT. ## Create an account and a machine user The [Getting Started](../../dev-guide/gettingstarted.html) page documents the required steps to get a hold of the `clientId`, `clientSecret` and `partitionId`. ## Create a token Use the values described in the [Authorization](../../dev-guide/howto.html) section to construct the Create Token request. ## Make API requests This guide will take you through the steps you need to perform to extract your energy consumption, emission, or cost data for a location or a meter in the granularity and for the period of interest. - Identify MediumConsumption ID - Read Consumption per building - Read Consumption per meter equipment > **Note for usage:**\ > Currently, we cannot identify the point which is streamed to Navigator. Extension of equipment types required to identify `Interface PointId` Assumption: Today, you manually assign one point to the meter equipment. → PointId can be identified via Equipment endpoints. # Changelog The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). ## [1.1.0-rc1] - 2026-05-18 #### IMPORTANT CHANGES - **API Endpoint Update:** New domain and API path for the Fire API. - **Old Endpoint:** `https://api.bpcloud.siemens.com/fire/*` - **New Endpoint:** `https://eu.buildingx.siemens.com/api/openness/fire/*` - **Action Required:** Please update your integrations to use the new endpoint. - **Grace Period:** The old domain will continue to be supported for a grace period. A separate notification will be sent to all customers regarding the deprecation date ### [1.0.3] - 2025-10-06 #### Removed - Commands API endpoint ### [1.0.2] - 2025-07-31 #### Updated - Enforce commanding for geofence enabled ### [1.0.1] - 2025-06-06 - Enable location API - Add new points API ### [1.0.0] - 2025-05-27 #### Fix - deprecate locations API ### [0.0.20] - 2025-05-15 #### Updated - Update response body of Configuration API ### [0.0.19] - 2025-03-12 #### Added - Site Information and Channels API, supports retrieving channels data along with its dynamic data ### [0.0.18] - 2024-11-05 #### Added - New Commands API ### [0.0.16] - 2024-11-05 - Added linting ### [0.0.15] - 2024-11-05 - Added Parent CustomerText Level 1, 2, 3 Attributes for the events API ### [0.0.14] - 2024-09-25 - revert deprecate locations API ### [0.0.13] - 2024-09-24 - Added fire.api-manager.openapi.yaml to the package in the dist subfolder ### [0.0.12] - 2024-09-23 - Enhance Fire API offering for events by adding event source as response ### [0.0.11] - 2024-08-08 - Aligned the documentation and corrected spelling errors ### [0.0.10] - 2023-05-28 - deprecate locations API ### [0.0.7] - 2023-09-28 #### Updated - added panel device example for the /device API ### [0.0.6] - 2023-07-21 #### Updated - remove page[limit] query parameter for the /device API ### [0.0.5] - 2023-07-15 #### Updated - Update Base url path of fire API for the developer portal ### [0.0.3] - 2023-06-27 #### Updated - Update env configuration for Fire APIs ### [0.0.1] - 2023-05-26 #### Added - First version of 'Fire API' # Fire API The Fire API enables you or rather the calling entity, the machine user to get all events of site if it has access to the partition of the site. ## Concepts & Glossary | Term | Description | | -------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Location | A structure or part of a structure where a Device can be located | | Device | A Device can be any electronic equipment with some computing ability that has a firmware or supports the installation of software. Examples are pumps, dampers, valves, sensors, detectors, limit switches, remote/local switches or automation devices. | | Event | An Event represents a non-normal (non-quiescent) state in building automation systems and is caused by an abnormal situation (technical: like fault, alarm, detector exclusion, range violation but also human: manually creating an alarm/task request/call). | ## Devices Some devices are connected directly to the Building X cloud and some make use of other devices to manage the connection. When a device relies on another device to connect to the cloud, it has a relationship called `hasGateway` defined, referring to the device it uses as a gateway to Building X. # Quick Start Welcome to the Fire API Quick Start guide! This document will walk you through the essential steps to authenticate and interact with the Fire APIs, from account creation to making your first API requests. ## Prerequisites Before you begin, ensure you have: - Access to the Fire API portal. - Credentials for a machine user (`clientId`, `clientSecret`, and `partitionId`). - A terminal environment (Linux/MacOS shell or Windows command prompt). - [curl](https://curl.se/) installed, or another HTTP client of your choice. ## Step 1: Create an Account and Machine User To access the Fire APIs, you need to create an account and register a machine user. Follow the instructions in the [Getting Started](../../dev-guide/gettingstarted.html) guide to obtain your `clientId`, `clientSecret`, and `partitionId`. ## Step 2: Obtain a JSON Web Token (JWT) Authenticate your machine user by generating a JWT. This token will be used to authorize your API requests. ### Example: Generate a Token Set your credentials as environment variables: ```bash export CLIENT_ID= export CLIENT_SECRET= ``` **Endpoint:** `POST https://siemens-bt-015.eu.auth0.com/oauth/token` **Body Parameters:** - `client_id` (string, required): Your machine user's client ID. - `client_secret` (string, required): Your machine user's client secret. - `audience` (string, required): The audience for the token, typically the API identifier. - `grant_type` (string, required): The OAuth2 grant type, usually `client_credentials`. **Example:** ```bash curl https://siemens-bt-015.eu.auth0.com/oauth/token \ -H 'content-type: application/json' \ -d "{ \"client_id\":\"$CLIENT_ID\", \"client_secret\":\"$CLIENT_SECRET\", \"audience\":\"https://horizon.siemens.com\", \"grant_type\":\"client_credentials\" }" ``` > **Note:** > > - The above example uses a Linux/MacOS shell. On Windows, use `set` instead of `export`. > - You can use any HTTP client or programming language to make these requests. #### Example Response ```json { "access_token": "eyJ0eXAiOiUSJ9.eyJpc3MiOiJdGlhbHMifQ.MJpcxLfyOt", "token_type": "Bearer", "expires_in": 86400 } ``` - The `access_token` is your JWT. - The `expires_in` value (in seconds) indicates how long the token is valid (typically 24 hours). Set your token and partition as environment variables for convenience: ```bash export PARTITION= export TOKEN= ``` ## Step 3: Make API Requests With your token and partition ID, you can now interact with the Fire API. ### 3.1 List Locations Retrieve all buildings (locations) within your partition. **Endpoint:** `GET partitions/{$partitionId}/locations` **Path Parameters:** - `partitionId` (string, required): The unique identifier of the partition. **Example:** ```bash curl -H "Authorization: Bearer $TOKEN" \ "https://eu.buildingx.siemens.com/api/openness/fire/partitions/$PARTITION/locations" ``` - The response includes a list of locations, each with an `id` property. - For large partitions, refer to the [Pagination](../../dev-guide/index.html#pagination) section. Store a location ID for later use: ```bash export LOCATION= ``` ### 3.2 Get Location by ID Retrieve details for a specific location. **Endpoint:** `GET partitions/{$partitionId}/locations/{$locationId}` **Path Parameters:** - `partitionId` (string, required): The unique identifier of the partition. - `locationId` (string, required): The unique identifier of the location. **Example:** ```bash curl -H "Authorization: Bearer $TOKEN" \ "https://eu.buildingx.siemens.com/api/openness/fire/partitions/$PARTITION/locations/$LOCATION" ``` - Replace `$LOCATION` with the desired location ID. ### 3.3 List Devices List all devices in your selected building. **Endpoint:** `GET partitions/{$partitionId}/devices` **Path Parameters:** - `partitionId` (string, required): The unique identifier of the partition. **Example:** ```bash curl -H "Authorization: Bearer $TOKEN" \ "https://eu.buildingx.siemens.com/api/openness/fire/partitions/$PARTITION/devices" ``` - Devices are associated with locations (buildings). ### 3.4 List Events Retrieve past events and alarms for a specific device. You can filter events based on timestamp ranges using query parameters. **Endpoint:** `GET partitions/{$partitionId}/devices/{$deviceId}/events` **Path Parameters:** - `partitionId` (string, required): The unique identifier of the partition. - `deviceId` (string, required): The unique identifier of the device. **Query Parameters:** - `filter[timestamp][from]` (string, optional): Start of the timestamp range (ISO 8601 format). - `filter[timestamp][to]` (string, optional): End of the timestamp range (ISO 8601 format). **Example:** ```bash curl -G -H "Authorization: Bearer $TOKEN" \ --data-urlencode "filter[timestamp][from]=2024-05-01T00:00:00Z" \ --data-urlencode "filter[timestamp][to]=2024-05-31T23:59:59Z" \ "https://eu.buildingx.siemens.com/api/openness/fire/partitions/$PARTITION/devices/$DEVICE/events" ``` - Replace the timestamp values with your desired date and time range in ISO 8601 format. - The response will include events within the specified time window. ### 3.5 Get Location Configuration Retrieves the configuration details for a specific location. This endpoint allows clients to fetch settings and parameters associated with a given location, such as thresholds, enabled features, and other configuration elements. **Endpoint:** `GET partitions/{$partitionId}/locations/{$locationId}/configuration` **Path Parameters:** - `partitionId` (string, required): The unique identifier of the partition. - `locationId` (string, required): The unique identifier of the location. **Example:** ```bash curl -H "Authorization: Bearer $TOKEN" \ "https://eu.buildingx.siemens.com/api/openness/fire/partitions/$PARTITION/locations/$LOCATION/configuration" ``` ### 3.6 Get Last Values for Points Fetch the most recent values for specified points. This endpoint is useful for monitoring and displaying the latest state or value of various points. **Endpoint:** `GET partitions/{$partitionId}/points/lastValues` **Path Parameters:** - `partitionId` (string, required): The unique identifier of the partition. **Query Parameters:** - `pointIds` (string, required): Comma-separated point IDs to retrieve the last values for. **Note:** You can specify up to 50 point IDs per request. - `lastChangedLatest` (string, optional): Only return the latest value for each point up to and including this timestamp (ISO 8601 format). **Example:** ```bash curl -G -H "Authorization: Bearer $TOKEN" \ --data-urlencode "pointIds=," \ --data-urlencode "lastChangedLatest=2024-06-01T12:00:00Z" \ "https://eu.buildingx.siemens.com/api/openness/fire/partitions/$PARTITION/points/lastValues" ``` - Replace ``, ``, etc., with the desired point IDs (up to 50 per request). - Set `lastChangedLatest` to the latest timestamp you want to include (optional). - The response will include the latest values for each specified point, up to the given timestamp if provided. #### Response The response contains an array of objects, each representing the last value for a requested point. Each object includes at least the following fields: - `id`: The point ID. - `value`: The latest value for the point. - `createdAt`: The timestamp when the value was recorded. Depending on the point type, the `value` and additional fields may vary. Below are examples of possible responses for different point types: Automatic Detector Test ```json { "id": "1fd1dd0c-c696-41cd-9898-1386f5cde725", "value": "TEST_SUCCESSFUL", "createdAt": "2021-03-08T13:51:40.541Z" } ``` Possible `value`: `TEST_SUCCESSFUL`, `TEST_FAILED`, `NOT_REACHABLE` Possible `failureDetails`: `COMPENSATION`, `GRID`, `COVER` Manual Test Activation ```json { "id": "1fd1dd0c-c696-41cd-9898-1386f5cde725", "createdAt": "2021-03-08T13:51:40.541Z", "value": "NORMAL" } ``` Possible `value`: `NORMAL`, `TEST` Automatic External Alarm Indicator Test ```json { "id": "1fd1dd0c-c696-41cd-9898-1386f5cde725", "createdAt": "2021-03-08T13:51:40.541Z", "value": "TEST_SUCCESSFUL" } ``` Possible `value`: `TEST_SUCCESSFUL`, `TEST_FAILED`, `TEST_DISABLED`, `NOT_REACHABLE`, `NO_EAI_CONFIGURED` Operating Years ```json { "id": "1fd1dd0c-c696-41cd-9898-1386f5cde725", "createdAt": "2021-03-08T13:51:40.541Z", "value": "14.32" } ``` Value is a number in years. Beam Compensation Value ```json { "id": "1fd1dd0c-c696-41cd-9898-1386f5cde725", "createdAt": "2021-03-08T13:51:40.541Z", "value": "82" } ``` Value is a number in percent. Compensation Value ```json { "id": "1fd1dd0c-c696-41cd-9898-1386f5cde725", "createdAt": "2021-03-08T13:51:40.541Z", "value": "82" } ``` Value is a number in percent. Compensation Level ```json { "id": "1fd1dd0c-c696-41cd-9898-1386f5cde725", "createdAt": "2021-03-08T13:51:40.541Z", "value": "MEDIUM" } ``` Possible `value`: `LOW`, `MEDIUM`, `HIGH` Grid Value ```json { "id": "1fd1dd0c-c696-41cd-9898-1386f5cde725", "createdAt": "2021-03-08T13:51:40.541Z", "value": "10" } ``` Value is a number. Grid Level ```json { "id": "1fd1dd0c-c696-41cd-9898-1386f5cde725", "createdAt": "2021-03-08T13:51:40.541Z", "value": "MINOR" } ``` Possible `value`: `NO`, `MINOR`, `MAJOR`, `FAULT` Sensitivity ```json { "id": "1fd1dd0c-c696-41cd-9898-1386f5cde725", "createdAt": "2021-03-08T13:51:40.541Z", "value": 12 } ``` Value is a number. > **Note:**\ > The response of the lastValue API can be any of these types, and each response will always include at least `value`, `createdAt`, and `id` fields. Refer to the API documentation for the full list of supported point types and value formats. **Example:** ```bash curl -G -H "Authorization: Bearer $TOKEN" \ --data-urlencode "pointIds=," \ --data-urlencode "lastChangedLatest=2024-06-01T12:00:00Z" \ "https://eu.buildingx.siemens.com/api/openness/fire/partitions/$PARTITION/points/lastValues" ``` - Replace ``, ``, etc., with the desired point IDs (up to 50 per request). - Set `lastChangedLatest` to the latest timestamp you want to include (optional). - The response will include the latest values for each specified point, up to the given timestamp if provided. > **Note:**\ > For more information on deprecation policies, pagination, filtering, and error handling, see the [Developer's Guide](../../dev-guide/gettingstarted.html). ## Visual Overview ## Configuration API usage ______________________________________________________________________ You are now ready to start building with the Fire API! For advanced usage, troubleshooting, and best practices, consult the [Developer's Guide](../../dev-guide/gettingstarted.html). # Changelog The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). ### [1.0.13] - 2026-05-5 #### IMPORTANT CHANGES - **API Endpoint Update:** New domain and API path for the Geometry API. - **Old Endpoint:** `https://api.bpcloud.siemens.com/geometry/v1/*` - **New Endpoint:** `https://eu.buildingx.siemens.com/api/openness/geometry/v1/*` - **Action Required:** Please update your integrations to use the new endpoint. - **Grace Period:** The old domain will continue to be supported for a grace period. A separate notification will be sent to all customers regarding the deprecation date ### [1.0.11] - 2024-08-08 #### Added - Aligned the documentation and corrected spelling errors ### [1.0.9] - 2024-04-16 #### Added - First version of Geometry API # Building Geometry API The Building Geometry API enables you to create and interact with 2D geometry of building floorplans. This API is organized around the `Floorplan`, and `Geometry` resources. A floorplan contains geometry in the form of a GeoJSON FeatureCollection. ## Concepts & Glossary | Term | Description | | --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Floorplan | A Floorplan represents meta data about the 2D geometry of a certain floor. This meta data included the angle (transformation) and centerpoint (origo) of the contained geometries. A floorplan relates to a `Floor` as defined in Structure API. | | Geometry | Is a GeoJSON FeatureCollection where each Feature represents either a `Room`, `RoomSegment`, `FloorArea` or `Equipment`, as defined in Structure API. | # Quick Start Getting started with using Building Geometry API involves the following steps: 1. Create an account and a machine user. 1. Create a JSON Web Token (JWT) by using the machine user credentials. 1. Make API requests using the JWT. Note In the following examples we are: 1. making use of a Linux/MacOS shell in which environmental variables are set using the `export` command. In other environments it may be different, e.g. Windows uses the `set` command instead. 1. using the `curl` as a client. But the API can be used in any programming language with an HTTP Client, e.g. Go, Python, NodeJS, Javascript and Java. ## Create an account and a machine user The [Getting Started](../../dev-guide/gettingstarted.html) page documents the required steps to get a hold of the `clientId`, `clientSecret` and `partitionId`. ## Create a token Use the values described in the [Authorization](../../dev-guide/howto.html) section to construct the Create Token request. ### Example request ```bash export CLIENT_ID= export CLIENT_SECRET= curl https://siemens-bt-015.eu.auth0.com/oauth/token \ -H 'content-type: application/json' \ -d "{ \"client_id\":\"$CLIENT_ID\", \"client_secret\":\"$CLIENT_SECRET\", \"audience\":\"https://horizon.siemens.com\", \"grant_type\":\"client_credentials\" }" ``` To run this example yourself, set the `CLIENT_ID` and `CLIENT_SECRET` first. ### Example response ```json { "access_token": "eyJ0eXAiOiUSJ9.eyJpc3MiOiJdGlhbHMifQ.MJpcxLfyOt", "token_type": "Bearer", "expires_in": 86400 } ``` The token, or JWT (JSON Web Token), is the value of the `access_token` property in the response. You can now use it by passing it in the `Authorization` header of any subsequent API requests. The `expires_in` property represents the number of seconds your token is valid, usually, the value corresponds to 24 hours. When this time has elapsed you will need to create a new token. Now you have all you need to start using the API. As a last step of preparation set the token and `partitionId` as environmental variables. As an option, you can find your `partitionId` by using the Accounts API. ```bash export PARTITION= export TOKEN= ``` ## Make API requests This guide will take you through the steps you need to perform to retrieve the 2D geometry of a floor. As a prerequisite, it is recommended to lookup a `floorId` either in the Data Setup application or using the Structure API. In the rest of this document we assume that the `floorId` is `20cfac0b-d3ae-415e-af95-861d46e5fdda`. ### Find Floorplan The first step to perform is to list the floorplans in your partition and specifying a filter for your floor. This you can do by performing the `List Floorplans` operation and a filter for the the `representsFloor.data.id` property. ```bash curl -H "Authorization: Bearer $TOKEN" \ "https://eu.buildingx.siemens.com/api/openness/geometry/partitions/$PARTITION/floorplans?filter[representsFloor.data.id]=20cfac0b-d3ae-415e-af95-861d46e5fdda" ``` The response contains all floorplans for the specified floor (floorId). If you have a large number of buildings you may need to retrieve multiple pages, see [Pagination](../../dev-guide/index.html#pagination). Select the `id` property of one of the floorplans in the response and set it in an environmental variable. E.g. if the `id` is `8db4216d-61c5-4e79-8558-164aa179bfe9` then set it using the following command: ```bash export FLOORPLAN_ID=8db4216d-61c5-4e79-8558-164aa179bfe9 ``` ### Get Geometry The next step is to fetch the geometry available in your floorplan. This is achieved by using the `List Geometry` operation. ```bash curl -H "Authorization: Bearer $TOKEN" \ "https://eu.buildingx.siemens.com/api/openness/geometry/partitions/$PARTITION/floorplans/$FLOORPLAN_ID/geometry" ``` The response contains a GeoJSON-document that can be processed by your application or rendered by multiple commercial and open source components or online tools. # Changelog The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). ## [2.0.6] - 2025-02-06 ### Added - Server URLs updated ## [2.0.5] - 2024-11-08 ### Added - Added POST method for tasks - Required select header for the GET, PUT and DELETE methods ## [2.0.2] - 2024-07-16 ### Added - Fixed spelling issues ## [2.0.1] - 2024-07-10 ### Added - Fixed typos - Fixed lint issues ## [2.0.0] - 2024-03-22 ### Added - Renamed to `Lifecycle Twin API` - Cleaned up the references from the invalid methods - Restructured the Endpoints ## [1.0.0] - 2023-06-22 ### Added - First version of 'Ecodomus API' # Lifecycle Twin API The Lifecycle Twin API enables you to create and manage various objects in the Lifecycle Twin. The Lifecycle Twin API is organized around: - Portfolio, Companies, Sites, Buildings - User, User Groups, Roles - Assets, Documents, Jobs - Model, View, Import, Export - Issues, Tags, Surveys ## Concepts & Glossary | Term | Description | | ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Portfolio | A customer subscription to Lifecycle Twin. Brings together customer's configurations, sites and building data. | | Company | A company describes the contact information of the organization. A company basically tied to the user or used as a manufacturer for an equipment. | | Site | A site is a grouping of multiple Buildings and Outside spaces. Typically it is physically represented by a fence, vegetation or street. It could be a contiguous area like a fenced-in airport or a looser aggregation like a government district (consisting of multiple individual buildings). | | Document | A document is generally a file which is directly related to a site, building or other assets. A document includes drawings, operations manuals, instructions, floor plans and others. Documents may contains 3D models as well. | | Building | A building is generally a structure built for a purpose that has a roof and walls and stands more or less permanently in one place. Typical examples are a house, factory, or shop; but if necessary also a cave, tower, tent or even a ship or bridge could be called a building. | | Zone | Zones represent the where aspect of building automation, combining functional and spatial aspects. Examples are comfort zones, fire alarm zones or lighting zones. | | Floor | A floor is a horizontal / level part of a building with a surface that could be used by people (for living, work, storage, recreation, etc), depending on context, they are also called storey/story, level or deck. Multiple floors can be stacked up vertically to form a multi-floor building. | | Room | A room is an indoor space enclosed by walls (and a roof or ceiling) to which entry is possible by a door or similar, which connects it either to a passageway, to another room, or to the outdoors. Normally, rooms are large enough for several persons to enter and move about or conduct the intended activity. | | Room Segment | Often, a floor is implicitly divided into a grid or into segments that later on might turn into rooms by adding walls. Such segments often contain technical preparations for possible rooms (e.g. pillars, grids, window arrangement, piping or wiring) | | Type | Describes common specifications of a component like physical dimension, color, manufacturer, etc. Typically it provides definition of a model. Type doesn't represents a physical instance. | | Component | Physical instance of a device or equipment. It can be any object like Wall, Pipe, Pipe Segment, Door, etc. | | Category | Its a property of a Component and Type which group them by functional purposes. | | System | A System contains a machinery and heavy equipment installed for the operation of a service. A system consists of partial systems, aggregates and components (equipment). | | Sub System | A part of a System. Represents smaller section of it. | | Task | Based on the use case a task represents an issue, work order or task related with a component, room or document or a registration of an event or emergency in a building or construction site. All tasks have a workflow and use responsible to the contribution of it. | | Inspection | An inspection is an order to execution of examination of a component or space. Inspection is characterized by digital form needs to be populated on execution. | | Survey | A functionality that enables the collection of information about a component on a mobile device via the use of digital forms for the purpose of collecting general information to solicit information during construction or operations. Surveys are created on the web desktop application and executed on mobile application. | | Reminder | An escalation of the overdue tasks to generate downloadable PDF reports of statuses and details of each task | # Quick start ## Get the API key Contact with the Lifecycle Twin support to retrieve the CLIENT_ID and CLIENT_SECRET. These to key are required for the authentication via API ## Authenticate Before making any calls to the Lifecycle Twin API need to authenticate and retrieve the `access_token` ### Request ```html POST /api/token HTTP/1.1 Host: eu-ecodomus-services.siemens.com Content-Type: application/x-www-form-urlencoded client_id=your_id&client_secret=your_secret&username=username&password=pwd&grant_type=password ``` ### Response ```json { "access_token": "PELPIm2Mn1-JzbZRqrUxa.............ICGvNGR-Y", "token_type": "bearer", "expires_in": 604799, "refresh_token": "2a66b5d105e141d.....2b80a8b7b35ea5bc91", "user_sid": "8b6c047f-0000-0000-0000-1ceb7331eaf7" } ``` For the further calls you need to use `access_token` See the [Authentication](guide/authentication.html) page for details. ```text curl -X POST \ https://eu-ecodomus-services.siemens.com/api/token \ -H 'Content-Type: application/x-www-form-urlencoded' \ -d 'client_id=[CLIENT_ID]&client_secret=[CLIENT_SECRET]&grant_type=password&username=[LOGIN]&password=[PASSWORD]' ``` ## Fetch the data Lifecycle Twin API offers methods to receive a list of objects of a particular type, and methods for receiving a particular object of the specified type using object’s unique ID. The following headers are required within HttpHeader: **Select** – this header contains a list of fields that need to be populated ```json {"Id", "Name", "CustomFields" : {"Id", "Value"}} ``` This request will return an entity or a list of entities which will have fields Id, Name, and a list of custom fields – CustomFields, where each field from CustomFields will have fields Id, Value populated. Note Currently the API does not guarantee populating ONLY those fields which are specified in the header. In some cases, other fields will be also populated. But the fields specified in the header will be populated. **Filter** – contains filter description. This filter will be used to receive a list of objects of a particular type. It is used only in the request for a list of objects. ```json {"Contains" : {"Name" : "100W - 277V"}} ``` A request with this Select header will return objects where Name field contains a value “100W – 277V”. A list of fields available for filtering is defined by the object schema. *Sample* ```json {"equals":{"CustomFields[4be891a7-58c8-4f14-8b5b-e878fe3016f3].Value":3}} ``` A request with this Select header will return objects where field Value contains a value “3” for CustomField with identifier equal to `4be891a7-58c8-4f14-8b5b-e878fe3016f3`. # Example of animation definition ## C# Classes to serialize to JSON ```csharp ``` ### JSON Sample ```json { "Animations": [ { "Conditions": [ { "RDSVar": "D-227A (level m3)", "Type": "Less", "Value1": 22, "Value2": null } ], "LogicalOperation": "OR", "AnimationType": "Color", "Parameters": "{\"Speed\":1.0,\"Color\":\"#00FF00\"}", "IsBlinked": false }, { "Conditions": [ { "RDSVar": "D-227A (level m3)", "Type": "Range", "Value1": 22, "Value2": 24 } ], "LogicalOperation": "OR", "AnimationType": "Color", "Parameters": "{\"Speed\":5.0,\"Color\":\"#FF9900\"}", "IsBlinked": true }, { "Conditions": [ { "RDSVar": "D-227A (level m3)", "Type": "Greater", "Value1": 24, "Value2": null } ], "LogicalOperation": "OR", "AnimationType": "Color", "Parameters": "{\"Speed\":10.0,\"Color\":\"#FF0000\"}", "IsBlinked": true } ], "Models": [ { "Id": "98cccd8e-feaa-4927-ad86-34db0b7357b9", "Ids": [ "c5578d57-a838-43f7-b942-0f0e5ddcde12-000c755e" ] } ] } ``` # Model animations Animations allows to visualize state of equipment in a model by applying the visual effects to a model object. There are following animation types are supported: - **Color**. Colors an object with the given color. Optionally object can blink with the given frequency. - **Outline**. Draw an object outline with the given color, width, intensity and blurring of the outline. Optionally can blink with the given frequency. Outline can be set to be visible through other objects. Also requires more client hardware resources. Not recommended to use too much outlines especially with the complicated geometry objects - **Rotation**. Rotates object clockwise be the vertical axis. ## Get list of animations ### Request **GET** `/models/animations/raw` | Parameter | Description | | ----------------- | ------------------------------------------------------ | | **clientname** | (*mandatory*) Portfolio name | | **Authorization** | (*mandatory*) Authentication token | | **select** | (*optional*) List of fields to include into the result | | **filter** | (*optional*) Filter parameters | | **Sort** | (*optional*) Sorting parameters | ### Response sample ```json [ { "Name": "Test animation", "Definition": "{\"Models\":[{\"Id\":\"f6e60649-aea1-4c3b-a627-0bf83f00b3f8\",\"Ids\":[\"68e3adeb-69f1-4edd-9fea-ebebe475ab3d-001843f8\",\"68e3adeb-69f1-4edd-9fea-ebebe475ab3d-001843f4\",\"68c03d5f-e7b7-47a0-8abb-9f13b8991453-00169e77\"]}],\"Animations\":[{\"Conditions\":[{\"RDSVar\":\"\",\"Type\":\"Equals\",\"Value1\":true}],\"LogicalOperation\":\"AND\",\"AnimationType\":\"Color\",\"Parameters\":\"{\\\"Speed\\\":10.0,\\\"Color\\\":\\\"#1E90FF\\\"}\",\"IsBlinked\":true}]}", "Id": "5abb3958-7b71-46ca-8eba-8b4143e65d21" }, { "Name": "Pump state", "Definition": "{\"Models\":[{\"Id\":\"f6e60649-aea1-4c3b-a627-0bf83f00b3f8\",\"Ids\":[\"68e3adeb-69f1-4edd-9fea-ebebe475ab3d-00184401\"]}],\"Animations\":[{\"Conditions\":[{\"RDSVar\":\"HSS_A0_L1009_0000ITP_PE09_PV\",\"Type\":\"Greater\",\"Value1\":10}],\"LogicalOperation\":\"AND\",\"AnimationType\":\"Color\",\"Parameters\":\"{\\\"Speed\\\":10.0,\\\"Color\\\":\\\"#1E90FF\\\"}\",\"IsBlinked\":true},{\"Conditions\":[{\"RDSVar\":\"HSS_A0_L1009_0000ITP_PE09_PV\",\"Type\":\"Greater\",\"Value1\":50}],\"LogicalOperation\":\"AND\",\"AnimationType\":\"Color\",\"Parameters\":\"{\\\"Speed\\\":20.0,\\\"Color\\\":\\\"#0000FF\\\"}\",\"IsBlinked\":true}]}", "Id": "1bee937f-aec6-4579-8284-b50149bf63aa" } ] ``` ## Create animation **POST** `/models/animations/raw` ### Payload example ```json { "Name": "Pump state", "Definition": "{\"Models\":[{\"Id\":\"f6e60649-aea1-4c3b-a627-0bf83f00b3f8\",\"Ids\":[\"68e3adeb-69f1-4edd-9fea-ebebe475ab3d-00184401\"]}],\"Animations\":[{\"Conditions\":[{\"RDSVar\":\"HSS_A0_L1009_0000ITP_PE09_PV\",\"Type\":\"Greater\",\"Value1\":10}],\"LogicalOperation\":\"AND\",\"AnimationType\":\"Color\",\"Parameters\":\"{\\\"Speed\\\":10.0,\\\"Color\\\":\\\"#1E90FF\\\"}\",\"IsBlinked\":true},{\"Conditions\":[{\"RDSVar\":\"HSS_A0_L1009_0000ITP_PE09_PV\",\"Type\":\"Greater\",\"Value1\":50}],\"LogicalOperation\":\"AND\",\"AnimationType\":\"Color\",\"Parameters\":\"{\\\"Speed\\\":20.0,\\\"Color\\\":\\\"#0000FF\\\"}\",\"IsBlinked\":true}]}" } ``` ## Modify animation **PUT** `/models/animations/raw/{animationId}` ### Payload example ```json { "Name": "Pump state", "Definition": "{\"Models\":[{\"Id\":\"f6e60649-aea1-4c3b-a627-0bf83f00b3f8\",\"Ids\":[\"68e3adeb-69f1-4edd-9fea-ebebe475ab3d-00184401\"]}],\"Animations\":[{\"Conditions\":[{\"RDSVar\":\"HSS_A0_L1009_0000ITP_PE09_PV\",\"Type\":\"Greater\",\"Value1\":10}],\"LogicalOperation\":\"AND\",\"AnimationType\":\"Color\",\"Parameters\":\"{\\\"Speed\\\":10.0,\\\"Color\\\":\\\"#1E90FF\\\"}\",\"IsBlinked\":true},{\"Conditions\":[{\"RDSVar\":\"HSS_A0_L1009_0000ITP_PE09_PV\",\"Type\":\"Greater\",\"Value1\":50}],\"LogicalOperation\":\"AND\",\"AnimationType\":\"Color\",\"Parameters\":\"{\\\"Speed\\\":20.0,\\\"Color\\\":\\\"#0000FF\\\"}\",\"IsBlinked\":true}]}", "Id": "1bee937f-aec6-4579-8284-b50149bf63aa" } ``` # Assemblies ## Get type assemblies **GET** /types/{typeId}/assemblies Returns list of assemblies for the given type **Output example**: ```json [ { "Name": "AC Units", "Items": [], "Id": "277831b4-915d-4c1a-9a71-f061706eb571" } ] ``` ## Create type assembly **POST** /types/{typeId}/assemblies Create assembly for the given type **Body**: ```json { "Name": "AC Units", } ``` ## Get type assembly items **GET** /types/{typeId}/assemblies/{assemblyId}/items *Returns list of items for the given assembly\** **Output example**: ```json [ { "Type": { "Name": "Air Conditioner (Outdoor Unit)", "IsMajor": false, "Specifications": [], "Assets": [], "AssetsCount": 0, "Id": "f9a85305-a7c3-495d-a23e-be60dcabf33c" }, "Id": "5297a696-35e0-46a8-b5a9-60b25d5dc1f0" }, { "Type": { "Name": "Air Conditioner (Indoor Unit)", "IsMajor": false, "Specifications": [], "Assets": [], "AssetsCount": 0, "Id": "6a525842-b1fb-457d-988c-aaeb7c1b0864" }, "Id": "b8f8b69d-db90-4c02-adb8-b6f6c28fe63e" } ] ``` ## Create type assembly item **POST** /types/{typeId}/assemblies/{assemblyId}/items Creates an assembly item of the given assembly **Body**: ```json { "Type": { "Id": "f9a85305-a7c3-495d-a23e-be60dcabf33c" }, "AssemblyType": { "Id": "18A57A89-F5C4-488C-82CF-8BB0BDEBAD3E" } } ``` ## Get asset assemblies **GET** /assets/{assetId}/assemblies Returns list of assemblies for the given asset **Output example**: ```json [ { "Name": "AC Units", "Items": [], "Id": "277831b4-915d-4c1a-9a71-f061706eb571" } ] ``` ## Get asset assembly items **GET** /assets/{assetId}/assemblies/{assemblyId}/items *Returns list of items for the given assembly\** **Output example**: ```json [ { "Type": { "Name": "Air Conditioner (Outdoor Unit)", "IsMajor": false, "Specifications": [], "Assets": [], "AssetsCount": 0, "Id": "f9a85305-a7c3-495d-a23e-be60dcabf33c" }, "Asset": { "Name": "AC-OU-1", "LocationsCount": 0, "Id": "657915b9-f680-4363-9442-d0cea4a2551c" }, "Id": "5297a696-35e0-46a8-b5a9-60b25d5dc1f0" }, { "Type": { "Name": "Air Conditioner (Indoor Unit)", "IsMajor": false, "Specifications": [], "Assets": [], "AssetsCount": 0, "Id": "6a525842-b1fb-457d-988c-aaeb7c1b0864" }, "Id": "b8f8b69d-db90-4c02-adb8-b6f6c28fe63e" } ] ``` ## Link assets to the assembly **PUT** /assets/{assetId}/assemblies/{assemblyId}/items/{assemblyItemId} Creates the linkup of asset to a given assembly item. The asset must be relevant to the assembly item type **SELECT** {"Asset"} **Body**: ```json { "Asset": { "Id": "657915b9-f680-4363-9442-d0cea4a2551c" } } ``` # Manage tab fields ## Fetch tab fields The method returns the fields, associated with the given tab **GET** /{entityName}/tabs/{tabId} **Output example**: ```json [ { "Order": 0, "Field": { "Name": "Employee Code", "IsSystem": false, "IsReadonly": false, "MaxLength": 0, "AllowMultipleValues": false, "Id": "00000000-0000-0000-0000-000000000000" }, "IsRequired": false, "IsHidden": false, "Id": "b447f15d-2998-48d1-8302-b24b0b6acd22", "EntityType": 0 }, { "Order": 1, "Field": { "Name": "Employee Name", "IsSystem": false, "IsReadonly": false, "MaxLength": 0, "AllowMultipleValues": false, "Id": "00000000-0000-0000-0000-000000000000" }, "IsRequired": false, "IsHidden": false, "Id": "02135de9-bbb9-40e0-b078-d2c4ec6bc0d7", "EntityType": 0 }, { "Order": 2, "Field": { "Name": "Trade", "IsSystem": false, "IsReadonly": false, "MaxLength": 0, "AllowMultipleValues": false, "Id": "00000000-0000-0000-0000-000000000000" }, "IsRequired": false, "IsHidden": false, "Id": "58530794-b70f-45a1-8064-97b2cda482cf", "EntityType": 0 } ] ``` ## Assign a field to a tab The method links a field to a given custom tab **POST** /{entityName}/tabs/{tabid}/fields **Body**: ```json { "Order": 2, "Field": { "Id": "8b773b85-a6cd-4dc8-bdb5-941983f27599" } } ``` Order property defines the order of the field in a tab. **Return value** Id of the created linkup ```json { "Value": "aeebf9fa-6926-4530-b064-687fc83c5c00" } ``` ## Unassign field from a tab Unlink the field from a custom tab **DELETE** /{entityName}/tabs/{tabId}/fields/{linkupId} # Manage tab value ## Get values of custom tab Returns the values of given entity for a custom tab **GET** /{entityName}/{entityId}/tabs/{tabId}/values **Output example**: ```json [ { "Id": "1a97acd9-27af-4ba6-b4f3-431f6c39a44d", "Fields": [ { "Id": "8b773b85-a6cd-4dc8-bdb5-941983f27599", "Name": "Trade" }, { "Id": "d37272b1-7546-499b-a47d-e1e3c4ce03c4", "Name": "Hours worked" }, { "Id": "14c06acc-5379-45e1-a0ea-5d4bdbbc81cc", "Name": "Employee Code" }, { "Id": "f1c4124f-0795-40b7-9048-6112a86dfbe8", "Name": "Employee Name" } ], "Values": [ { "Record": [ "MAINT", 7.0, "ECO-STT", "Stan Tang" ] }, { "Record": [ "MAINT", 2.0, "ECO-LHT", "Lenny Hong" ] }, { "Record": [ "MAINT", 5.0, "ECO-SCC", "Suise Chaing" ] } ] } ] ``` ## Set the values for a custom tab The method sets the values for a given tab. The method replace the entire set of data for a tab. To clean the value need to call the method with the empty values **POST** /{entityName}/{entityId}/tabs/{tabId}/values **Body**: ```json { "Id": "1a97acd9-27af-4ba6-b4f3-431f6c39a44d", "Fields": [ { "Id": "14c06acc-5379-45e1-a0ea-5d4bdbbc81cc" }, { "Id": "f1c4124f-0795-40b7-9048-6112a86dfbe8" }, { "Id": "8b773b85-a6cd-4dc8-bdb5-941983f27599" }, { "Id": "d37272b1-7546-499b-a47d-e1e3c4ce03c4" } ], "Values": [ { "Record": [ "ECO-STT", "Stan Tang", "MAINT", 7 ] }, { "Record": [ "ECO-LHT", "Lenny Hong", "MAINT", 2 ] }, { "Record": [ "ECO-SCC", "Suise Chaing", "MAINT", 5 ] } ] } ``` **Example**: # Manage tabs ## Get custom tabs for an entity Returns list of assemblies for the given type **GET** /{entityName}/tabs **Output example**: ```json [ { "Name": "Book Labor", "Order": 0, "EntityType": "issue", "Id": "1a97acd9-27af-4ba6-b4f3-431f6c39a44d" } ] ``` ## Create a new custom tab for an entity Create new custom tab for the given entity type **POST** /{entityName}/tabs **Body**: ```json { "Name" : "Permits", "Order" : 1 } ``` Order property defines the order of the tabs in the UI. **Return value** Id of the created tab ```json { "Value": "b3bf0c6f-1fcc-4e4b-9eae-48798e94d258" } ``` ## Update existing custom tab Update name or order of the existing custom tab **PUT** /{entityName}/tabs/{tabId} **Select**: `{"Name", "Order"}` **Body**: ```json { "Order" : 1, "Name" : "Permits*" } ``` ## Delete custom tab Delete existing custom tab by given id **DELETE** /{entityName}/tabs/{tabId} # Custom fields ## Get list of custom fields ### Request **GET** `/customfields` | Parameter | Description | | ----------------- | ------------------------------------------------------ | | **clientname** | (*mandatory*) Portfolio name | | **authorization** | (*mandatory*) Authentication token | | **select** | (*optional*) List of fields to include into the result | | **filter** | (*optional*) Filter parameters | | **sort** | (*optional*) Sorting parameters | ### Response sample ```json [ { "Name": "Address", "FieldType": { "Name": "Text", "DataType": 3, "Description": "The attribute data should be interpreted as a string of text.", "IsSystem": true, "Id": "110cc2e6-5cae-4e6c-a435-19fe184acee0" }, "IsSystem": false, "IsReadonly": false, "MaxLength": 0, "Id": "ba37e9b8-f7c4-4a21-a0c7-2154260ee732" } ] ``` ## Get list of custom field types ### Request **GET** `/customfields/types` | Parameter | Description | | ----------------- | ------------------------------------------------------ | | **clientname** | (*mandatory*) Workspace name | | **Authorization** | (*mandatory*) Authentication token | | **select** | (*optional*) List of fields to include into the result | | **filter** | (*optional*) Filter parameters | | **Sort** | (*optional*) Sorting parameters | ### Available fields ```json {"Id", "Name", "DataType", "Description", "Values": {"Id", "Name"}} ``` ### Response sample ```json [ { "Name": "Dropdown", "DataType": 3, "Values": [ { "Name": "Value 1", "Id": "141393a5-7fb3-4559-858f-ca1c2d231c7a" }, { "Name": "Value 2", "Id": "0aa6bffe-28d7-4418-8631-2152424bebfd" } ], "IsSystem": false, "Id": "4e004b9b-c474-4926-904b-d1dcc41a048c" } ] ``` ## Get list of custom field type values ### Request **GET** `/customfields/types/{fieldTypeId:guid}/values` ## Get object structure group **GET** `/{entityType}/groups` ## Get fields in the object structure group **GET** `/{entityType}/groups/{groupId}/fields` ## Get object structure group (for a category) **GET** `/categories/{categoryId}/{entityType}/groups` ## Get fields in the object structure group (for a category) **GET** `/categories/{categoryId}/{entityType}/groups/{groupId}/fields` # Create document **Endpoint** - `/documents/upload` HTTP Method – **POST** ## Request ### Headers | Parameter | Description | | ----------------- | ---------------------------------- | | **clientname** | (*mandatory*) Portfolio name | | **authorization** | (*mandatory*) Authentication token | | **projectId** | (*mandatory*) Project identifier | ### Body Type – form-data ### Keys Name: A, Type: Text, Value:JSON ,described the document object. The sample of the JSON can be retrieved using the Get/Find Documents method Name:B, Type: File, Value – document file ### Example of the Document JSON ```json { "Name": "2012-03-23-Duplex-Programming.xlsx", "Parent": { "Id": "723d3996-6b6d-4429-93bc-0d9bb4472e31" }, "Files": [ { "Size": 158064, "Name": "2012-03-23-Duplex-Programming.xlsx", "ContentType": "application/octet-stream" } ] } ``` Note - Id is a mandatory field where ID is the ID of a folder. Without Parent Id you will be able create a new document, but you will not find in the in use interface. - Files is an array, but you always must specify only one file - Need to pass the file size in bytes explicitly (will be solved in the future) ## Response HTTP status code 200 ### Body In the Response result there is Id of the created document Media type: application/json **Example**: ```json { "Value": "841054dd-ce63-4e93-b176-0753ff2ec1eb" } ``` # Create folder **Endpoint** - `/documents` HTTP Method – **POST** ## Request ### Headers | Parameter | Description | | ----------------- | ---------------------------------- | | **clientname** | (*mandatory*) Portfolio name | | **authorization** | (*mandatory*) Authentication token | | **projectId** | (*mandatory*) Project identifier | ### Body JSON of the folder definition **Media type** : application/json **Example:** ```json { "Name": "DWG Drawings", "Parent": { "Id": "723d3996-6b6d-4429-93bc-0d9bb4472e31" }, "IsFolder": **true** } ``` Note - Id is a mandatory field where ID is the ID of a folder. Without Parent Id you will be able create a new folder, but you will not find in the in use interface. - IsFolder: true is mandatory to create a folder ## Response HTTP status code 200 ### Body In the Response result there is Id of the created folder Media type: application/json **Example**: ```json { "Value": "9c2e4b64-ab91-470b-92ff-62d78263478e" } ``` # Document new version Lifecycle Twin API doesn't have a logic to create a new version automatically for the file with the same name. API provides the method to check if the new version must be created or not. Normally the process to create a new version requires 2 steps: - Check if the document with the same file name already exist - Upload the new file to the existing document ## Check if document exist Use the standard GET query with the relevant filter **GET** /{entityName}/tabs | Header | Value | | ---------- | ------------------------------------------------------------ | | **filter** | {"equals":{"Files.Name":"Screenshot 2022-12-20 180429.png"}} | **Output example**: ```json [ { "Name": "Screenshot 2022-12-20 180429.png1", "Files": [ { "Name": "Screenshot 2022-12-20 180429.png", "Size": 0, "SourceType": 0, "CreatedOn": "0001-01-01T00:00:00Z", "Version": 0, "TotalFileParts": 0, "FilePart": 0, "Id": "9eae91b1-0c89-8777-5d70-dcc238af074e" } ], "Version": 0, "Parent": { "Name": "RME_basic_sample_project", "Version": 0, "IsFolder": false, "SubFoldersCount": 0, "ChildrenCount": 0, "IsAutoName": false, "CheckedOut": false, "Id": "2320efac-0214-49ad-ae5d-a1a99eb5b6c6" }, "Id": "9c5dc8bd-57c9-4196-adc6-9ec2369df975" } ] ``` ## Upload new version **PUT** /documents/{documentId}/upload | Header | Value | | ------ | ------------------------------------------- | | select | {"Files": {"Id", "ContentType"}, "Version"} | **Body** - Multipart **First part**: ```json { "Id": "{documentId}", "Files": [ { "Name": "Screenshot 2022-12-20 180429.png", "ContentType": "image/png", "FilePart": 1 } ] } ``` Second part - file stream Note - Files.Name must be exactly the same as the name of the file in the document. Otherwise the new version will not be created - ContentType is mandatory and defined by the host application - FilePart is mandatory and equals 1 # Download document **Endpoint** - `/entities/{documentId}/files/{fileId}` HTTP Method – **GET** ## Request ### Headers | Parameter | Description | | ----------------- | ---------------------------------- | | **clientname** | (*mandatory*) Portfolio name | | **authorization** | (*mandatory*) Authentication token | | **projectId** | (*mandatory*) Project identifier | ## Response Response body will have the content of the downloaded file Note \*\*\*GET /documents\*\* endpoint can be used to get the documentId. Using *select* header, user can include *Files* to get the file details # Find document ## Get/Find Documents **Endpoint** - `/documents` HTTP Method **GET** The method returns list of documents or folders by specified criteria. ### Request #### Headers | Parameter | Description | | ----------------- | -------------------------------------------------------- | | **clientname** | (*mandatory*) Portfolio name | | **authorization** | (*mandatory*) Authentication token | | **projectId** | (*mandatory*) Project identifier | | **select** | (*optional*) List of fields to include into the result | | **filter** | (*optional*) Filter parameters | | **sort** | (*optional*) Sorting parameters | | **pagination** | (*optional*) Returns only the part of the requested data | #### Use cases Most uses cases to get documents and/or folder are solved by the proper filter 1. Get the root folder. All folders and files must be unloaded into the root folder. Need to get root folder id then pass this Id as a parent Id. ```json {"bool":{"must":[{"equals":{"IsFolder":true}}, {"is\empty": "Parent"}]}} ``` 1. Get subfolders of the folder ```json {"bool":{"must":[{"equals":{"IsFolder":true}}, {"equals": {"Parent.Id": "723d3996-6b6d-4429-93bc-0d9bb4472e31"}}]}} ``` 1. Find a document by name (in the entire project) ```json {"bool":{"must":[{"equals":{"IsFolder":false}}, {"equals": {"Name": "230500-AHU C1 Performance Data.pdf"}}]}} ``` 1. Find a document by name (in the specific folder) ```json {"bool":{"must":[{"equals":{"IsFolder":false}}, {"equals": {"Parent.Id": "723d3996-6b6d-4429-93bc-0d9bb4472e31"}}, {"equals": {"Name": "230500-AHU C1 Performance Data.pdf"}}]}} ``` 1. Find the documents with filtering by custom filed ```json {"bool":{"must":[{"equals":{"CustomFields[f6194a0f-4d4b-451d-9eb0-62d24f58b72c].Value":"A1"}},{"equals":{"Name":"03-Maintenance Guide A"}}]}} ``` 1. Find the documents updated after a certain time ```json {"bool":{"must":[{"equals":{"IsFolder":false}}, {"greater\than\or\equals": {"ModifiedOn": "2021-03-26T13:19:29.683Z"}}]}} ``` # Modify document ## Modify document metadata **Endpoint** - `/documents/` HTTP Method – **PUT** ### Request #### Headers | Parameter | Description | | ----------------- | ------------------------------------------------------ | | **clientname** | (*mandatory*) Portfolio name | | **authorization** | (*mandatory*) Authentication token | | **projectId** | (*mandatory*) Project identifier | | **select** | (*optional*) List of fields to include into the result | #### Body JSON of the document metadata **Media type** : application/json **Example**: ```json { "Id": "841054dd-ce63-4e93-b176-0753ff2ec1eb", "Name": "2012-03-23-Duplex-Programming.xlsx", "Category": { "Id": "734e44c5-fcf8-e011-b8c2-00101832264b" } } ``` Note - The ID in the JSON and ID in the endpoint must be the same - The **Select** header is used to specify what properties must be updated. This approach allows you to specify only the fields you want to update without passing other properties. In case if one or another property is not updated, please check it is included to the select - You cannot specify that custom field must be updated. In case if you update one custom field, need to specify "CustomFields" in the header and pass all custom field values in JSON. #### Response HTTP status code 204 # Data points Formerly RDS Variables ## Fetch Data points The method returns the list of data points for a given portfolio **GET** /variables ### Output example ```json [ { "Name": "CHS_M1_L3003_CVS0011_000_0000_AM_2.1", "Type": 0, "Comment": "Module 1.T1.AHU1.1. Alarm 2", "Rds": { "Name": "Simulation", "Host": "http://aeuc1c015009a.ad003.siemens.net/simulated-rds/", "Value": 0, "Port": 0, "Enable": true, "Id": "6297c24b-76c6-44dc-825d-92d9d6b2e68f" }, "StringValue": "False", "Id": "ef811a85-da4b-400f-bcaa-9e4351423855" }, { "Name": "CHS_M1_L3003_CVS0011_000_0000_AM_2.10", "Type": 0, "Comment": "Module 1.Т1.AHU1.1. Alarm 2", "Rds": { "Name": "Simulation", "Host": "http://aeuc1c015009a.ad003.siemens.net/simulated-rds/", "Value": 0, "Port": 0, "Enable": true, "Id": "6297c24b-76c6-44dc-825d-92d9d6b2e68f" }, "StringValue": "True", "Id": "d8ea9c4c-bc00-4e4b-a3f1-3369d74fd483" } ] ``` For the CRUD operations use the relevant POST, GET, PATCH and DELETE methods. # RDS ## Fetch RDS The method returns the list of remote data services for a given portfolio **GET** /rds ### Output example ```json [ { "Name": "Simulation", "Host": "http://aeuc1c015009a.ad003.siemens.net/simulated-rds/", "Value": 0, "Port": 0, "Enable": true, "Id": "6297c24b-76c6-44dc-825d-92d9d6b2e68f" } ] ``` ## Fetch devices The method returns the list of devices for the given RDS **GET** /rds/{rdsId}/remotedevices ### Output example ```json [ { "Id": "268bd558-360b-46b5-9d27-c125e8f20103", "Name": "A0_4ShUO1_502", "Options": { "IP": "10.98.208.28", "Port": 502, "Unit": 0, "HighByteFirst": false, "HighRegisterFirst": false, "UseGroupRead": false, "UseServerCache": false, "DevicePort": 0, "Net": 0, "Timeout": 0, "Retries": 0 }, "Protocol": 11 }, { "Id": "aaaa0461-c3cd-4557-b942-fe30c2854220", "Name": "A0_4ShUO1_502****", "Options": { "IP": "10.98.208.41", "Port": 502, "Unit": 0, "HighByteFirst": false, "HighRegisterFirst": false, "UseGroupRead": false, "UseServerCache": false, "DevicePort": 0, "Net": 0, "Timeout": 0, "Retries": 0 }, "Protocol": 11 } ] ``` ### List of protocols for device ```csharp public enum Protocol { OPCDA = 1, MODBUSRTU = 2, MODBUSTCP = 3, BACNETIP = 4, BACNETTCP = 5, BACNETMSTP = 6, LOCALDB = 7, SMARTCONNECTOR = 8, PI_WEB = 9, MQTT = 10, MODBUSTCP2 = 11, /// /// SmartConnector (Single Tag Value Request) REST protocol /// SMARTCONNECTOR_SR = 12, } ``` ## Fetch tags The method returns the list of devices linked with the given device. Note The method **doesn't** return the list of all available tags on this device **GET** /rds/{rdsId}/remotedevices/{deviceId} ### Output example ```json [ { "Id": "2fea9f56-7498-4d61-b0be-195e91528894", "Name": "DGP_A0_4ShUO001_00_p788", "IODeviceId": "268bd558-360b-46b5-9d27-c125e8f20103", "Address": "1/3/12788", "Type": 13, "ObjectType": 0, "EndZeroScale": 0.0, "EndFullScale": 0.0, "RawZeroScale": 0.0, "RawFullScale": 0.0, "IsFolder": false }, { "Id": "e2d88815-a7db-4204-96b4-e70d6802bc34", "Name": "DGP_A0_4ShUO001_00_p789", "IODeviceId": "268bd558-360b-46b5-9d27-c125e8f20103", "Address": "1/3/12789", "Type": 13, "ObjectType": 0, "EndZeroScale": 0.0, "EndFullScale": 0.0, "RawZeroScale": 0.0, "RawFullScale": 0.0, "IsFolder": false } ] ``` ### List of types for tag ```csharp public enum RemoteTagType { /// /// Boolean type: TRUE to FALSE /// BOOL = 1, /// /// Byte type: 0 to 255 /// BYTE, /// /// Byte type: -128 to 127 /// SBYTE, /// /// Unicode 16 bit character: U+0000 to U+FFFF /// CHAR, /// /// Decimal, 28-29 significant digits: (-7.9 x 10^28 to 7.9 x 10^28) / 10^(0 to 28) /// DECIMAL, /// /// Double, 15-16 digits: +-5.0*10^-324 to +-1.7*10^308 /// DOUBLE, /// /// Float, 7 digits: -3.4*10^38 to +3.4*10^38 /// FLOAT, /// /// Signed integer, 32 bit value: -2 127 483 648 to 2 147 483 647 /// INT, /// /// Unsigned integer, 32 bit value: 0 to 4 294 967 295 /// UINT, /// /// Signed 64 bit integer: - 9 223 372 036 854 775 808 to 9 223 372 036 854 775 807 /// LONG, /// /// Unsigned 64 bit integer: 0 to 18 446 774 073 709 551 615 /// ULONG, /// /// General type for all others /// OBJECT, /// /// Signed 16 bit integer: -32768 to 32767 /// SHORT, /// /// Unsigned 16 bit integer: 0 to 65535 /// USHORT, /// /// Zero or more Unicode characters /// STRING } ``` For the CRUD operations use the relevant POST, GET, PATCH and DELETE methods. # Assets The section describes the main methods to manage Assets ## Types **Endpoint** - `/types` The endpoint is used to manage Types. Specification and schema can be found in the [API Reference](../api-reference.html) ### Additional endpoints **Endpoint** - `/types/{typeId}/assemblies` Manages assemblies of a given type. More details you can find in the [Assemblies](../examples/assemblies/assemblies.html) page ## Components **Endpoint** - `/assets` The endpoint is used to manage Component. Specification and schema can be found in the [API Reference](../api-reference.html) ### Additional endpoints **/assets/{assetId}/systems** - To manage systems of a component with the given ID. **/assets/{assetId}/assemblies** - To manage assemblies of a given component. More details you can find in the [Assemblies](../examples/assemblies/assemblies.html) page ## Rooms **Endpoint** - `/location` The endpoint is used to manage Rooms & segments. ## Systems **Endpoint** - `/systems` The endpoint is used to manage systems. ## Zones **Endpoint** - `/zone` The endpoint is used to manage Zones. ### Additional endpoints **/zones/{zoneId}/locations** - To manage spaces of a zone with a given ID Specification and schema can be found in the [API Reference](../api-reference.html) ## Floors **Endpoint** -- `/floors` The endpoint is used to manage Floors. Specification and schema can be found in the [API Reference](../api-reference.html) ## Building **Endpoint** - `/facilities` The endpoint is used to manage Buildings. Specification and schema can be found in the [API Reference](../api-reference.html) # Authentication Lifecycle Twin API is using Json Web Token (JWT) for authentication. The tokens are created by the server, signed with a secret key and passed over to the client, which is using this token for authentication. Call the `api/token` endpoint to obtain the token The token endpoint is found at: {{ApiUrl}}/token The following parameter must be include to the **Body**: | Parameter | Description | | ----------------- | ---------------------------------------------------------------------- | | **client_id** | (*mandatory*) The client ID provided by Lifecycle Twin Support | | **client_secret** | (*mandatory*) The client secret key provided by Lifecycle Twin Support | | **grant_type** | (*mandatory*) password | | **username** | (*mandatory*) Your Lifecycle Twin username | | **password** | (*mandatory*) Your Lifecycle Twin password | Response sample: ```json { "access_token": "I-Iy4rCh2zTVQlk.....vMKExs-9tjjF4zgvKm9IunuAg", "token_type": "bearer", "expires_in": 604799, "refresh_token": "254120b05398496689d3de9754def62a781be8028d9d40549d4d46e310d4132f", "user_sid": "87fbd0ac-3230-4839-a5a8-3eb9f96acdea" } ``` # Create POST is used to send data to the server to create/update a resource. The data sent to the server with POST is stored in the request body of the HTTP request. **POST** `/clientprojects` ```bash POST /api/clientprojects HTTP/1.1 clientname: authorization: ``` ***Request Body*** ```json { "Name": "My Project", "Latitude": 57, "Longitude": 13, "OwnerOrganization": { "Id": "bb601d80-0c8d-4a4d-bece-230ad4b12d9c" }, "LeadOrganization": { "Id": "bb601d80-0c8d-4a4d-bece-230ad4b12d9c" }, "Location": { "Id": "af9d605a-fae6-4ed4-b123-d37dea32f0c3" } } ``` ***Response*** The response will have Status Code 200 (OK) if the resource successfully created. Response body will have {Id} of the created resource. # Delete The DELETE method deletes the specified resource. The {id} of the resource to be deleted needs to specified as a query parameter **DELETE** `/clientprojects/{id}` ```bash PUT /api/clientprojects HTTP/1.1 clientname: authorization: select: {"Name", "Latitude", "Longitude"} ``` ***Response*** The response will have Status Code 204 (No Content) if the request accepted. # Querying data The GET method is used to query information from Lifecycle Twin. GET method without {id} parameter returns all the resources of the requested type. In the example below, the GET API returns all the Sites (Projects) from the requested Portfolio (Client). Note The **clientname** and **authorization** HTTP headers are mandatory for all the API requests. Refer [Quick Start](../quickstart.html) to generate **authorization** token. **GET** `/clientprojects` ```bash GET /api/clientprojects HTTP/1.1 clientname: authorization: select: {"Name", "Id", "CreatedOn", "CityName", "ModifiedOnLocal","OwnerOrganization", "Latitude", "Longitude", "Location": {"Id", "Name"}} ``` In addition to the mentioned headers, **projectid** HTTP header is also mandatory while interacting with Site specific resources. **GET** `/documents` ```bash GET /api/clientprojects HTTP/1.1 clientname: authorization: projectid: select: {"Name", "Id", "CreatedOn", "CityName", "ModifiedOnLocal","OwnerOrganization", "Latitude", "Longitude", "Location": {"Id", "Name"}} ``` ## Examples ### Query all sites **GET** `/clientprojects` ```bash GET /api/clientprojects HTTP/1.1 clientname: authorization: select: {"Name", "Id", "CreatedOn", "CityName", "ModifiedOnLocal","OwnerOrganization", "Latitude", "Longitude", "Location": {"Id", "Name"}} ``` GET method with {id} parameter returns a specific resource which matches the {id} value. ### Get information of a specific site by id **GET** `/clientprojects/2f4978fe-ab1c-4d66-be94-2086d0887a86` #### Request ```bash GET /api/clientprojects HTTP/1.1 clientname: authorization: select: {"Name", "Id", "CreatedOn", "CityName", "ModifiedOnLocal","OwnerOrganization", "Latitude", "Longitude", "Location": {"Id", "Name"}} ``` #### Response ```json [ { "Name": "Building1", "IsEnabled": false, "CreatedOn": "2022-10-18T14:32:17.023Z", "Location": { "Name": "Zug", "ChildrenCount": 0, "Id": "4dc758d7-ab1c-4f4b-bd42-fd95c3cd8236" }, "Id": "2f4978fe-ab1c-4d66-be94-2086d0887a86", "CreatedOnLocal": "2022-10-18T14:32:17.023+00:00" } ] ``` While querying the data, user can specify field selection, filtering, sorting and pagination using HTTP Headers ## Selecting User can specify the list of fields/ properties they would like to include in the response using *select* HTTP header to improve the performance and reduce the number of data to be transferred. Some of the fields are returned in the result, regardless if they specified in the *select* header or not. The good practice is do not include the fields you don't need for the certain scenario. Especially the referenced fields and collections. On the backend side it generates *JOIN* query, and in some cases may significantly impact to the performance. The *select* header supports fields of the nesting properties as well. The syntax of the select header is close to the JSON. ```json {"Field1", "Field2", "Field3": {"Field1OfField3", "Field2OfField3"}} ``` Use *select* header in all GET queries The list of available fields for an entity can be found in the [API Reference](../api-reference.html) In case of specifying an incorrect field, the exception will be thrown. ### Example of the request **GET** `/clientprojects` ```bash GET /api/clientprojects HTTP/1.1 clientname: authorization: select: {"Name", "Id", "CreatedOn", "CityName", "ModifiedOnLocal","OwnerOrganization", "Latitude", "Longitude", "Location": {"Id", "Name"}} ``` ### Response ```json [ { "Name": "Building1", "IsEnabled": false, "CreatedOn": "2022-10-18T14:32:17.023Z", "Location": { "Name": "Zug", "ChildrenCount": 0, "Id": "4dc758d7-ab1c-4f4b-bd42-fd95c3cd8236" }, "Id": "2f4978fe-ab1c-4d66-be94-2086d0887a86", "CreatedOnLocal": "2022-10-18T14:32:17.023+00:00" }, { "Name": "Building2", "IsEnabled": false, "CreatedOn": "2022-10-18T14:32:17.023Z", "Location": { "Name": "Zurich", "ChildrenCount": 0, "Id": "4dc758d7-xy1c-4f4b-bd42-fd95c3cd8236" }, "Id": "2f4978fe-xy1c-4d66-be94-2086d0887a86", "CreatedOnLocal": "2022-10-18T14:32:17.023+00:00" } ] ``` ## Filtering User can specify the filter condition to filter the objects using **filter** HTTP header. It is the powerful mechanism allow user to restrict the query results and find the required information. Filters are used not only for to search the objects, but also to fetch the related entities. For example, to get the list of components which belong to a room, the [/assets](assets.html) endpoint must be used with the filter by room (location) The following filter clauses are supported: | Clause | Description | | -------------------------- | ----------- | | **contains** | | | **any** | | | **ends_with** | | | **equals** | | | **greater_than** | | | **greater_than_or_equals** | | | **is_empty** | | | **less_than** | | | **less_than_or_equals** | | | **starts_with** | | | **in** | | Note Usage of proper clause may significantly affect to the performance. Use *equals* instead of *contains* may increase the performance up to 10 times. Correct clause based on your use cases. ### Simple filter by one property ```json {"clause":{"member":"value"}} {"equals":{"Name":"ISB-020-000--000_1000319"}} ``` ### Simple filter by one property of the referenced field ```json {"clause":{"Property.Field":"value"}} {"Equals":{"Systems.Id":"b7da56c7-f498-40f7-9104-80ba51cd62ba"}} ``` ### Filter by field of the collection property ```json {"clause":{"CollectionProperty[].Field":"value"}} {"equals":{"ModelObjects[].ObjectId":"41958798-cb80-4c0b-bfaf-63e53192c615-0007b076"}} ``` ### Example of the request **GET** `/clientprojects` ```html GET /api/clientprojects HTTP/1.1 clientname: authorization: filter: {"Contains" : {"Name" : "Building1"}} select: {"Name", "Id", "CreatedOn", "CityName", "Latitude", "Longitude", "Location": {"Id", "Name"}} ``` ## Sorting User can order your data in the response using **sort** HTTP header. ### Syntax ```json {"Property": "OrderType"} ``` **OrderType** must be *descending* or *ascending* Note It is required to use sorting if you fetch the data by pages. In case if you fetch all the data, it is required to use sorting by the unique field, for example, *Id*. Otherwise the data on the different pages can be duplicated and not all records will be returned. ### Example of the request **GET** `/clientprojects` ```html GET /api/clientprojects HTTP/1.1 clientname: authorization: filter: {"Contains" : {"Name" : "Building1"}} sort: {"Name": "descending"} select: {"Name", "Id", "CreatedOn", "CityName", "Latitude", "Longitude", "Location": {"Id", "Name"}} ``` ## Pagination User can specify pagination (page number and number of records per page) using the **pagination** HTTP header ### Syntax ```json {"page" : pageNumber, "size" : pageSize} ``` Note It is recommended do not use the large page size especially with the large number of referenced fields in the **select** header as it may impact to the performance. Normally, the page size should not exceed 1000 records ### Example of the request **GET** `/clientprojects` ```html GET /api/clientprojects HTTP/1.1 clientname: authorization: filter: {"Contains" : {"Name" : "Building1"}} sort: {"Name": "ascending"} select: {"Name", "Id", "CreatedOn", "CityName", "Latitude", "Longitude", "Location": {"Id", "Name"}} Pagination: {"page" : 0, "size" : 20} ``` The above request will return the first 20 records as response # Tasks The section describes the main methods to manage Tasks **Endpoint** - `/issues/` The main endpoint to manage Tasks. Refer to the [querying](querying.html), [creating](create.html) and [update](update.html) pages for more details ## Get activities of a tasks **GET** /bcf/issues/{issueId}}/comments ## Post a new comment into the task activity **POST** /bcf/issues/{issueId}/comments/upload The [multipart form-data](https://www.w3.org/TR/html401/interact/forms.html#h-17.13.4.2) is used to send the the payload for the comment. The first part contains the JSON payload of the comment, and the second part - attached image (optional) ### Example ```bash curl --location 'https://us-ecodomus-services.siemens.com/api/bcf/issues/d7ea5b6e-7ae8-4ea4-83a2-c2f78e903e18/comments/upload' \ --header 'clientname: ABC' \ --header 'Authorization: bearer 3Z0Qx...' \ --header 'projectId: d7b3ae7a-b72f-4430-aaaf-063cd3939b37' \ --form 'a="{\"Comment\": \"With Image\"}"' \ --form 'b=@"/D:/Downloads/image.png"' ``` ## Change task status **PUT** /workflows/issues/move/{issueId}/from/{statusFrom}/to/{statusTo} The endpoint change the status of the task with the given ID # Update PUT is used to send data to the server to create/update a resource. The data sent to the server with PUT is stored in the request body of the HTTP request. **PUT** `/clientprojects/{id}` ```bash PUT /api/clientprojects HTTP/1.1 clientname: authorization: select: {"Name", "Latitude", "Longitude"} ``` ***Request Body*** ```json { "Name": "My Project", "Latitude": 57, "Longitude": 13 } ``` ***Response*** The response will have Status Code 204 (No Content) if the request accepted. **Note:** The request body should only contain the fields which needs to be updated. No other properties should be included in the request body. The fields which needs to be updated should also be added in the "select" HTTP custom header. ## Remove/ Empty field value Removing/ emptying value of a fields can be done as below **PUT** `/clientprojects/{id}` ```bash PUT /api/clientprojects HTTP/1.1 clientname: authorization: select: {"Name", "CityName"} ``` ***Request Body*** ```json { "Name": "My Project - New", "CityName": "" } ``` # Changelog The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). ### [1.2.0] - 2026-04-28 #### IMPORTANT CHANGES - **API Endpoint Update:** New domain and API path for the Point Ingest API. - **Old Endpoint:** `https://api.bpcloud.siemens.com/ingest/*` - **New Endpoint:** `https://eu.buildingx.siemens.com/api/openness/ingest/*` - **Action Required:** Please update your integrations to use the new endpoint. - **Grace Period:** The old domain will continue to be supported for a grace period. A separate notification will be sent to all customers regarding the deprecation date ### [1.1.0] - 2026-04-27 #### Added - New endpoint `POST /points/covs` for multi-point batch value ingestion (up to 50 points per request) - Support for `covAttributes` and `targetValue` fields on point values - Multi-status (207) response for partial failure reporting ### [1.0.8] - 2025-03-17 #### Modify - Modify attribute for permissions ### [1.0.7] - 2024-08-09 #### Fixed - Fix response code ### [1.0.6] - 2024-08-08 #### Added - Aligned the documentation and corrected spelling errors ### [1.0.0] - 2023-11-10 #### Added - First version of 'Point Value Ingest API' # Point Value Ingest API The Point Value Ingest API enables you to set values of Points. The data that is measured by data points is called point values. The API provides two ingestion modes: - **Single-point ingestion:** Push 1 to 100 values for a single point using `POST /points/{pointId}/values`. - **Multi-point batch ingestion:** Push 1 value for up to 50 points in a single request using `POST /points/covs`. ## Concepts & Glossary | Term | Description | | ---------- | ---------------------------------------------------------------- | | PointValue | The data that is measured by data points is called point values. | # Quick Start Getting started with using add Point Value Ingest API involves the following steps: 1. Create an account and a machine user. 1. Create a JSON Web Token (JWT) by using the machine user credentials. 1. Make API requests using the JWT. Note In the following examples we are: ```text 1. making use of a Linux/MacOS shell in which environmental variables are set using the `export` command. In other environments it may be different, e.g. Windows uses the `set` command instead. 2. using the `curl` as a client. But the API can be used in any programming language with an HTTP Client, e.g. Go, Python, NodeJS, Javascript and Java. ``` ## Create an account and a machine user The [Getting Started](../../dev-guide/gettingstarted.html) page documents the required steps to get a hold of the `clientId`, `clientSecret` and `partitionId`. ## Create a token Use the values described in the [Authorization](../../dev-guide/howto.html) section to construct the Create Token request. ### Example request ```bash export CLIENT_ID= export CLIENT_SECRET= curl https://siemens-bt-015.eu.auth0.com/oauth/token \ -H 'content-type: application/json' \ -d "{ \"client_id\":\"$CLIENT_ID\", \"client_secret\":\"$CLIENT_SECRET\", \"audience\":\"https://horizon.siemens.com\", \"grant_type\":\"client_credentials\" }" ``` To run this example yourself, set the `CLIENT_ID` and `CLIENT_SECRET` first. ### Example response ```json { "access_token": "eyJ0eXAiOiUSJ9.eyJpc3MiOiJdGlhbHMifQ.MJpcxLfyOt", "token_type": "Bearer", "expires_in": 86400 } ``` The token, or JWT (JSON Web Token), is the value of the `access_token` property in the response.You can now use it by passing it in the `Authorization` header of any subsequent API requests.The `expires_in` property represents the number of seconds your token is valid, usually, the value corresponds to 24 hours. When this time has elapsed you will need to create a new token. Now you have all you need to start using the API. As a last step of preparation set the token and `partitionId` as environmental variables. ```bash export PARTITION= export TOKEN= ``` ## Make API requests This guide will take you through the steps you need to perform to add point values. ### Add point values To add point values, the `Post values` operation can be used. Push 1 to 100 values for given point in a batch. Only one pointValue per millisecond will be stored. PointValues are dropped if there are timestamp conflicts, i.e. if a pointValue with the same timestamp already exists. Example with following pointValues: - 2022-07-11T11:11:30.333 - 2022-07-11T11:11:30.222 - 2022-07-11T11:11:30.333 - 2022-07-11T11:11:30.444 The first is accepted (it's the first for time "30.333"). The second is also accepted. The third is dropped because it's the same timestamp as the first. The fourth is accepted again. The indices of dropped pointValues are listed in the response (0-based), in this example there is one conflict with index "2". If there is no conflict status 204 is returned instead of 202. Note: Use query parameter 'overwrite=true' to force overwriting of conflicting values. ```bash curl -X 'POST' \ "https://eu.buildingx.siemens.com/api/openness/ingest/partitions/$PARTITION/points/$POINTID/values" \ -H "Accept: application/vnd.api+json" \ -H "Authorization: Bearer $TOKEN" \ -H "Content-Type: application/vnd.api+json" \ -d "{ \"data\": [ { \"type\": \"PointValue\", \"attributes\": { \"timestamp\": \"2000-05-11T10:32:54.964Z\", \"value\": \"12.66\" } } ] }" ``` In the response, you will find the values added for the pointId. ### Add point values for multiple points To add point values for multiple points at once, use the `POST /points/covs` endpoint. Push 1 time-series value for 1 to 50 points in a batch. Existing values are always overwritten. ```bash curl -X 'POST' \ "https://eu.buildingx.siemens.com/api/openness/ingest/partitions/$PARTITION/points/covs" \ -H "Accept: application/vnd.api+json" \ -H "Authorization: Bearer $TOKEN" \ -H "Content-Type: application/vnd.api+json" \ -d "{ \"data\": [ { \"type\": \"Point\", \"id\": \"$POINTID_1\", \"attributes\": { \"values\": { \"type\": \"PointValue\", \"attributes\": { \"timestamp\": \"2000-05-11T10:32:54.964Z\", \"value\": \"12.66\" } } } }, { \"type\": \"Point\", \"id\": \"$POINTID_2\", \"attributes\": { \"values\": { \"type\": \"PointValue\", \"attributes\": { \"timestamp\": \"2000-05-11T10:32:54.964Z\", \"value\": \"22.50\" } } } } ] }" ``` A successful request returns status 204 (No Content). If some points fail while others succeed, a 207 (Multi-Status) response is returned with details about unprocessed entries and errors. Note For more details on deprecation policies and common API features such as paging, filtering and errors, refer to the [Developer's Guide](../../dev-guide/gettingstarted.html). ## Changelog The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). ### [1.4.0] - 2026-05-6 #### IMPORTANT CHANGES - **API Endpoint Update:** New domain and API path for the Security PIAM API. - **Old Endpoint:** `https://api.bpcloud.siemens.com/sec-piam-v1/*` - **New Endpoint:** `https://eu.buildingx.siemens.com/api/openness/sec-piam-v1/*` - **Action Required:** Please update your integrations to use the new endpoint. - **Grace Period:** The old domain will continue to be supported for a grace period. A separate notification will be sent to all customers regarding the deprecation date ### [1.3.3] - 2025-03-18 ### Bug Fixes - add missing error codes ([d084f46](https://code.siemens.com/horizon/api-manager/security-api/security-api-piam-specification/commit/d084f4670fdae3b80376669f3b0b14c5a4cc1ec7)) ### [1.3.2] - 2024-12-23 #### Revision - Updated changelog.md ### [1.3.1] - 2024-11-14 #### Revision - Aligned the changelog ### [1.3.0] - 2024-09-25 #### Features - I add properties to identity api ([24ec0a5](https://code.siemens.com/horizon/api-manager/security-api/security-api-piam-specification/commit/24ec0a5aed7e712e65876358c3d3a60f57ddca6d)) ### [1.2.0] - 2024-09-04 #### Added - Include `externalId` attribute in privileges retrieval ### [1.1.1] - 2024-08-08 #### Added - Aligned the documentation and corrected spelling errors ### [1.0.0] - 2023-10-20 #### Added - Security Physical Identity and Access Management API released - Added API endpoints for - Identities - To perform Create, Update, Delete and Read identity - Privileges - Retrieve list of configure privileges - Assign privileges to an identity # Security Identities and Privileges API The Identities and Privileges API enables you to create and manage Identities and assign configured privileges to an identity. This enables customers to programmatically manage identities, privileges in Security Manager. Customers can use the API to integrate the Security Manager functions & data into their own workflows & applications. ## Concepts & Glossary | Term | Description | | --------- | ---------------------------------------------------------------------------------------------------------------------- | | Identity | Identity is a user in access control system. Customers can manager their users, assign access rights to their premise. | | Privilege | Privilege is a set of access rights to manage user access to the premises with specified period. | # Quick Start Getting started with using Security Identities and Privileges APIs involves the following steps: 1. Create an account and a machine user. 1. Create a JSON Web Token (JWT) by using the machine user credentials. 1. Make API requests using the JWT. Note In the following examples we are: 1. Making use of a Linux/MacOS shell in which environmental variables are set using the `export` command. In other environments it may be different, e.g., Windows uses the `set` command instead. 1. Using the `curl` as a client. But the API can be used in any programming language with an HTTP Client, e.g. Go, Python, NodeJS, JavaScript and Java. ## Create an account and a machine user The [Getting Started](../../../dev-guide/gettingstarted.html) page documents the required steps to get a hold of the `clientId`, `clientSecret` and `partitionId`. ## Create a token Use the values described in the [Authorization](../../../dev-guide/howto.html) section to construct the Create Token request. ### Example request ```bash export CLIENT_ID= export CLIENT_SECRET= curl https://siemens-bt-015.eu.auth0.com/oauth/token \ -H 'content-type: application/json' \ -d "{ \"client_id\":\"$CLIENT_ID\", \"client_secret\":\"$CLIENT_SECRET\", \"audience\":\"https://horizon.siemens.com\", \"grant_type\":\"client_credentials\" }" ``` To run this example yourself, set the `CLIENT_ID` and `CLIENT_SECRET` first. ### Example response ```json { "access_token": "eyJ0eXAiOiUSJ9.eyJpc3MiOiJdGlhbHMifQ.MJpcxLfyOt", "token_type": "Bearer", "expires_in": 86400 } ``` The token, or JWT (JSON Web Token), is the value of the `access_token` property in the response. You can now use it by passing it in the `Authorization` header of any subsequent API requests. The `expires_in` property represents the number of seconds your token is valid, usually, the value corresponds to 24 hours. When this time has elapsed, you will need to create a new token. Now you have all you need to start using the API. As a last step of preparation set the token and `partitionId` as environmental variables. ```bash export PARTITION= export TOKEN= ``` ## Make API requests This guide will take you through the steps you need to: 1. Consume and create identities 1. Consume privileges ### List Identities To fetch the user based by `PARTITION`. `size` parameter is an optional field and based on this value the response are returned (Note: `size` should not exceed 1000). `cursor` represent the last identity (identity from last page) `id` from which the next set of identities can be fetched along with the size parameter. In case all the identities returned on the current request, on the next request, the `next` will be same as `self` value with empty `data` returned as response. ```bash curl -H "Authorization: Bearer $TOKEN" \ -H "Content-Type: application/vnd.api+json" \ "https://eu.buildingx.siemens.com/api/openness/sec-piam-v1/partitions/$PARTITION/identities?page[size]=2" ``` The response contains all details of individual identities. ```bash { "links": { "self": "/identities?page[size]=2", "next": "/identities?page[size]=2&page[cursor]=7945" }, "data": [ { "id": 7944, "type": "Identity", "attributes": { "firstName": "Christoper", "lastName": "Adam", "email": "christoper.adam@siemens.com", "credentials": [ { "cardNumber": "12345", "id": "cf8112e0-3350-420b-b817-bc4bc9a58727", "validity": { "validForUnlimitedTime": true, "validFromUtc": "2024-03-13T00:00:00Z", "validToUtc": "0001-01-01T00:00:00Z" }, "active": false } ], "properties": { "departmentId": "R&D", "location": "Germany" } } }, { "id": 7945, "type": "Identity", "attributes": { "firstName": "Lucy", "lastName": "Clare", "email": "lucy.clare@siemens.com", "credentials": [ { "cardNumber": "123", "id": "bf8112e0-3350-420b-b817-bc4bc9a58727", "validity": { "validForUnlimitedTime": true, "validFromUtc": "2024-03-13T00:00:00Z", "validToUtc": "0001-01-01T00:00:00Z" }, "active": false } ] "properties": { "departmentId": "R&D", "location": "Germany" } } } ] } ``` ### Add Identity To add new identity. The request body should contain the `type` and `attributes` fields. Data inside `properties` field are dynamic and should be aligned with Identity Type selected for the customer. ```bash curl -X "POST" \ -H "Authorization: Bearer $TOKEN" \ -H "Content-Type: application/vnd.api+json" \ "https://eu.buildingx.siemens.com/api/openness/sec-piam-v1/partitions/$PARTITION/identities" ``` with request body ```bash { "type": "Identity", "attributes": { "firstName": "James", "lastName": "Smith", "email": "james.smith@siemens.com", "credentials": [{ "cardNumber": "1211", "validity": { "validForUnlimitedTime": true, "validFromUtc": "2024-03-13T00:00:00Z", "validToUtc": "0001-01-01T00:00:00Z" }, "active": false }], "properties": { "departmentId": "R&D", "location": "Germany" } } } ``` The response contains newly created identity. ```bash { "links": { "self": "/identities", }, "data": { "id": 7946, "type": "Identity", "attributes": { "firstName": "James", "lastName": "Smith", "email": "james.smith@siemens.com", "credentials": [{ "cardNumber": "1211", "id": "dc8112e0-3350-420b-b817-bc4bc9a58727", "validity": { "validForUnlimitedTime": true, "validFromUtc": "2024-03-13T00:00:00Z", "validToUtc": "0001-01-01T00:00:00Z" }, "active": false }] } } } ``` ### Update Identity To update an identity. The request body should contain the `type` and `attributes` fields. `attributes` field shall contain only the modified fields to be updated, its not required to include fields which are unchanged. If new credential is added or changes in existing credential, `credentials` field should have the complete data with all credentials details. Credentials field of the Identity will be replaced with the `credentials` field from the update request. `properties` field shall have only the modified property. ***If the identity is configured as Readonly Identity type, except `credentials` all other fields in the update request will be skipped*** sample request body ```bash { "type": "Identity", "attributes": { "lastName": "Tom", "email": "james.tom@siemens.com", "properties": { "departmentId": "Operation" } } } ``` ### List Privileges To fetch the privileges based by `PARTITION`. ```bash curl -H "Authorization: Bearer $TOKEN" \ -H "Content-Type: application/vnd.api+json" \ "https://eu.buildingx.siemens.com/api/openness/sec-piam-v1/partitions/$PARTITION/privileges" ``` The response contains all details of individual privileges. ```bash { "links": { "self": "/privileges" }, "data": [ { "id": 11423, "type": "Privilege", "attributes": { "name": "Privilege sample", "description": "Privilege description", "externalId" : "10" } }, { "id": 11424, "type": "Privilege", "attributes": { "name": "Privilege sample", "description": "Privilege description", "externalId" : "11" } } ] } ``` Note For more details on deprecation policies and common API features such as paging, filtering and errors, refer to the [Developer's Guide](../../../dev-guide/gettingstarted.html). # Changelog The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). ## [0.4.0] - 2026-05-01 #### IMPORTANT CHANGES - **API Endpoint Update:** New domain and API path for the Security Intrusion API. - **Old Endpoint:** `https://api.bpcloud.siemens.com/sec-intrusion/v1/*` - **New Endpoint:** `https://eu.buildingx.siemens.com/api/openness/sec-intrusion/v1/*` - **Action Required:** Please update your integrations to use the new endpoint. - **Grace Period:** The old domain will continue to be supported for a grace period. A separate notification will be sent to all customers regarding the deprecation date ### [0.3.0] - 2025-09-25 - Add field "LastConfigChangeUtc" to determine when the intrusion system configuration has changed ### [0.2.0] - 2025-07-28 - Add installation number ### [0.1.1] - 2025-05-19 #### Bug Fixes - Add permission to call system and fixed ### [0.0.9] - 2025-05-16 #### Bug Fixes - Use *partitionId* instead of *partition_id* ### [0.0.8] - 2025-05-08 #### Added - First version of Security Intrusion API # Security Intrusion API This REST API allows you to retrieve information about all cloud-enabled Intrusion systems. An Intrusion system consists of one or multiple on-premise control panels connected to various field devices, such as detectors and input/output devices. To enable cloud connectivity, you need to install the Siveillance Intrusion Edge App and configure it to connect to the desired Intrusion control panel. ## Concepts & Glossary | Term | Description | | ------------- | ----------------------------------------------------------------- | | System | An Intrusion System consisting of one or multiple control panels. | | Control Panel | A hardware controller where an Intrusion Firmware is running on. | # Quick Start Sample quickstart for Reference API Getting started with using Reference APIs involves the following steps: 1. Create an account and a machine user. 1. Create a JSON Web Token (JWT) by using the machine user credentials. 1. Make API requests using the JWT. Note In the following examples we are: 1. making use of a Linux/MacOS shell in which environmental variables are set using the `export`-command. In other environments it may be different, e.g. Windows uses the `set`-command instead. 1. using the `curl` as a client. But the API can be used in any programming language with an HTTP Client, e.g. Go, Python, NodeJS, Javascript and Java. ## Create an account and a machine user The [Getting Started](../../../dev-guide/gettingstarted.html) page documents the required steps to get a hold of the `clientId`, `clientSecret` and `partitionId`. ## Create a token Use the values described in the [Authorization](../../../dev-guide/howto.html)- section to construct the Create Token request. ### Example request ```bash export CLIENT_ID= export CLIENT_SECRET= curl {{env.issUrl}} \ -H 'content-type: application/json' \ -d "{ \"client_id\":\"$CLIENT_ID\", \"client_secret\":\"$CLIENT_SECRET\", \"audience\":\"{{env.audience}}\", \"grant_type\":\"client_credentials\" }" ``` To run this example yourself, set the `CLIENT_ID` and `CLIENT_SECRET` first. ### Example response ```json { "access_token": "eyJ0eXAiOiUSJ9.eyJpc3MiOiJdGlhbHMifQ.MJpcxLfyOt", "token_type": "Bearer", "expires_in": 86400 } ``` The token, or JWT (JSON Web Token), is the value of the `access_token`-property in the response. You can now use it by passing it in the `Authorization`-header of any subsequent API requests. The `expires_in`-property represents the number of seconds your token is valid, usually, the value corresponds to 24 hours. When this time has elapsed you will need to create a new token. Now you have all you need to start using the API. As a last step of preparation set the token and `partitionId` as environmental variables. ```bash export PARTITION= export TOKEN= ``` # Changelog The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). ### [1.2.0] - 2026-05-06 #### IMPORTANT CHANGES - **API Endpoint Update:** New domain and API path for the Security Monitoring API. - **Old Endpoint:** `https://api.bpcloud.siemens.com/sec-monitoring/v1/*` - **New Endpoint:** `https://eu.buildingx.siemens.com/api/openness/sec-monitoring/v1/*` - **Action Required:** Please update your integrations to use the new endpoint. - **Grace Period:** The old domain will continue to be supported for a grace period. A separate notification will be sent to all customers regarding the deprecation date ### [1.1.0] - 2026-04-17 #### Feature - Add new endpoint: `/accesses/per-hour` ### [1.0.0] - 2026-03-05 #### Feature - Release of Security Monitoring API # Security Monitoring API Security Monitoring API allows to retrieve data of access events. ## Endpoints - Accesses: - per-day: allows to retrieve the count of accesses of unique identities per day - per-hour: allows to retrieve the count of accesses of unique identities per hour ## Concepts & Glossary | Term | Description | | --------------- | --------------------------------------------------------------------------------------------------- | | Access event | Access granted event that is triggered by person | | Unique identity | Unique individual, identified by used credential. An identity can use different types of credential | # Quick Start Sample quickstart for Monitoring API Getting started with using Monitoring APIs involves the following steps: 1. Create an account and a machine user. 1. Create a JSON Web Token (JWT) by using the machine user credentials. 1. Make API requests using the JWT. Note In the following examples we are: ```text 1. making use of a Linux/MacOS shell in which environmental variables are set using the `export`-command. In other environments it may be different, e.g. Windows uses the `set`-command instead. 2. using the `curl` as a client. But the API can be used in any programming language with an HTTP Client, e.g. Go, Python, NodeJS, Javascript and Java. ``` ## Create an account and a machine user The [Getting Started](../../../dev-guide/gettingstarted.html) page documents the required steps to get a hold of the `clientId`, `clientSecret` and `partitionId`. ## Create a token Use the values described in the [Authorization](../../../dev-guide/howto.html)- section to construct the Create Token request. ### Example request ```bash export CLIENT_ID= export CLIENT_SECRET= curl {{env.issUrl}} \ -H 'content-type: application/json' \ -d "{ \"client_id\":\"$CLIENT_ID\", \"client_secret\":\"$CLIENT_SECRET\", \"audience\":\"{{env.audience}}\", \"grant_type\":\"client_credentials\" }" ``` To run this example yourself, set the `CLIENT_ID` and `CLIENT_SECRET` first. ### Example response ```json { "access_token": "eyJ0eXAiOiUSJ9.eyJpc3MiOiJdGlhbHMifQ.MJpcxLfyOt", "token_type": "Bearer", "expires_in": 86400 } ``` The token, or JWT (JSON Web Token), is the value of the `access_token`-property in the response. You can now use it by passing it in the `Authorization`-header of any subsequent API requests. The `expires_in`-property represents the number of seconds your token is valid, usually, the value corresponds to 24 hours. When this time has elapsed you will need to create a new token. Now you have all you need to start using the API. As a last step of preparation set the token and `partitionId` as environmental variables. ```bash export PARTITION= export TOKEN= ``` ## Make API requests This guide will take you through the steps you need to perform to retrieve Monitoring API endpoints. # Changelog The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). ## [1.1.0] - 2026-05-01 #### IMPORTANT CHANGES - **API Endpoint Update:** New domain and API path for the Self Service API. - **Old Endpoint:** `https://api.bpcloud.siemens.com/sec-workflow/v1/*` - **New Endpoint:** `https://eu.buildingx.siemens.com/api/openness/sec-workflow/v1/*` - **Action Required:** Please update your integrations to use the new endpoint. - **Grace Period:** The old domain will continue to be supported for a grace period. A separate notification will be sent to all customers regarding the deprecation date ### [1.0.0] - 2025-03-11 #### Added - v1 of Security Workflow API # Security Workflow API The Security Workflow API enables you to retrieve workflows and start workflow instances, which are the 2 resources of the API. ## Concepts & Glossary | Term | Description | | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------- | | Workflows | A generic definition of a process consisting of several tasks and actions. | | Workflow Instance | One specific instance of a workflow, tied to concrete actions and people. An instance is created using a workflow as a form of "blueprint". | # Quick Start Sample quickstart for Reference API Getting started with using Reference APIs involves the following steps: 1. Create an account and a machine user. 1. Create a JSON Web Token (JWT) by using the machine user credentials. 1. Make API requests using the JWT. **Attention** In the following examples we are: ```text 1. making use of a Linux/MacOS shell in which environmental variables are set using the `export`-command. In other environments it may be different, e.g. Windows uses the `set`-command instead. 2. using the `curl` as a client. But the API can be used in any programming language with an HTTP Client, e.g. Go, Python, NodeJS, Javascript and Java. ``` ## Create an account and a machine user The [Getting Started](../../../dev-guide/gettingstarted.html) page documents the required steps to get a hold of the `clientId`, `clientSecret` and `partitionId`. ## Create a token Use the values described in the [Authorization](../../../dev-guide/howto.html)- section to construct the Create Token request. ### Example request ```bash export CLIENT_ID= export CLIENT_SECRET= curl {{env.issUrl}} \ -H 'content-type: application/json' \ -d "{ \"client_id\":\"$CLIENT_ID\", \"client_secret\":\"$CLIENT_SECRET\", \"audience\":\"{{env.audience}}\", \"grant_type\":\"client_credentials\" }" ``` To run this example yourself, set the `CLIENT_ID` and `CLIENT_SECRET` first. ### Example response ```json { "access_token": "eyJ0eXAiOiUSJ9.eyJpc3MiOiJdGlhbHMifQ.MJpcxLfyOt", "token_type": "Bearer", "expires_in": 86400 } ``` The token, or JWT (JSON Web Token), is the value of the `access_token`-property in the response. You can now use it by passing it in the `Authorization`-header of any subsequent API requests. The `expires_in`-property represents the number of seconds your token is valid, usually, the value corresponds to 24 hours. When this time has elapsed you will need to create a new token. Now you have all you need to start using the API. As a last step of preparation set the token and `partitionId` as environmental variables. ```bash export PARTITION= export TOKEN= ``` ## Make API requests This guide will take you through the steps you need to perform to call and use the Security Workflow API. ### Workflows Before you can create a workflow instance, you need to get the `Id` of the respective workflow. You can do so, by executing the following request: ```sh curl -H "Authorization: Bearer $TOKEN" "{{buildingxopenness.env.selfServiceApiUrl}}/partitions/$PARTITION/api/v1/workflows?showInSsp=true" ``` #### Example Response ```text [ { "id": "dd52c89a-e5b3-49fe-9f7c-6c72ee5ac0d8", // you will need this in the next step "name": "Your Workflow", "description": "Your Workflow's description", "active": true } ] ``` You might see a translation key as a name and description, depending on the configuration. Please keep in mind you will need to include the query parameter. If you don't, you will likely receive a `403 Forbidden`. ### Workflow Instances Once you have obtained the `Id` of the workflow you want to start, find out which payload the workflow requires, which is dependant on the process' definition. Set them as the environmental variable `PAYLOAD` (as shown in the curl). They have to be a JSON-object, which can contain any content. Then, do the following request: ```sh curl -X POST -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" -d '{ "workflowId": "dd52c89a-e5b3-49fe-9f7c-6c72ee5ac0d8", "instanceVariables": $PAYLOAD }' "{{buildingxopenness.env.selfServiceApiUrl}}/partitions/$PARTITION/api/v1/workflow-instances" ``` This will return a `204 No Content` if everything went as expected. Keep in mind that errors in the workflow itself will not be visible at this point. If there's a mistake with the content of the `instanceVariables` you will not be informed by the API. # Changelog The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). ## [1.0.4] - 2026-05-01 #### IMPORTANT CHANGES - **API Endpoint Update:** New domain and API path for the Streaming API. - **Old Endpoint:** `https://api.bpcloud.siemens.com/streaming/v1/*` - **New Endpoint:** `https://eu.buildingx.siemens.com/api/openness/streaming/v1/*` - **Action Required:** Please update your integrations to use the new endpoint. - **Grace Period:** The old domain will continue to be supported for a grace period. A separate notification will be sent to all customers regarding the deprecation date ### [1.0.2] - 2026-03-03 #### Update - Update description. ### [1.0.0] - 2026-03-01 #### Added - Release of Streaming API # Streaming API Streaming API for real-time data streams for points. ## Concepts & Glossary | Term | Description | | -------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Point Stream | A Point Stream determines the real-time data flow for specific sensor points (e.g., temperature, pressure) from IoT devices to registered webhook endpoints over a defined period. | | Webhook Registration | Webhook Registration is the process where clients register HTTP endpoints to receive real-time notifications for specific point data events. | | Event Publishing | Event Publishing is the separate process where the Point Vertical system publishes sensor data to SQS queues for processing and delivery to registered webhooks. | | Partition | A Partition represents a logical grouping or tenant boundary that isolates streaming subscriptions and events for different customers or organizational units. | | Stream Subscription | A Stream Subscription defines which specific points a webhook endpoint wants to monitor, including filtering criteria and delivery preferences. | | Webhook Secret | A Webhook Secret is a shared key used to authenticate and verify the integrity of webhook deliveries using HMAC signatures for secure communication. | | Dead Letter Queue | A Dead Letter Queue (DLQ) stores failed webhook delivery attempts for manual review and reprocessing when endpoints are temporarily unavailable. | # Quick Start Getting started with using Streaming APIs involves the following steps: 1. Obtain a JSON Web Token (JWT) for your machine user role. 1. Register webhook endpoints for real-time notifications. 1. Test webhook deliveries with sample events. > **Note:** In the following examples we are: > > 1. Making use of a Linux/MacOS shell in which environmental variables are set using the `export`-command. In other environments it may be different, e.g. Windows uses the `set`-command instead. > 1. Using the `curl` as a client. But the API can be used in any programming language with an HTTP Client, e.g. Go, Python, NodeJS, Javascript and Java. ## Authentication o use the Streaming API, you need a valid JWT token with the required permissions. Please visit [Authentication](../../dev-guide/howto.html) section to construct the Create Token request. ## Using Your Token Once you have obtained your JWT token, set it as an environment variable along with your partition ID and webhook configuration: ```bash export PARTITION= export TOKEN= export WEBHOOK_URL= export WEBHOOK_SECRET= # Optional: for webhook signature verification ``` You can now use the token by passing it in the `Authorization` header of all API requests. ## Make API requests This guide will take you through the steps you need to perform to register webhooks and start receiving real-time notifications. ### Register Point Stream Webhook Before you can receive point data notifications, you need to register a webhook for specific sensor points. You can do so by executing the following request: ```sh curl -X POST \ -H "Authorization: Bearer $TOKEN" \ -H "Content-Type: application/json" \ -d '{ "webhookUrl": "https://test-api.com/test/webhook", "pointIds": [ "88d45219-a0cf-4509-8f3d-84d15e6a9c44" ], "webhookSecret": "webhook-secret-key" }' \ "https://https://eu.buildingx.siemens.com/api/openness/streaming/v1/partitions/$PARTITION/streams?type=point-stream" ``` **Note:** The `webhookSecret` field is optional. If omitted, webhook payloads will not include signature verification. #### Example Response ```json { "id": "12345678-1234-1234-1234-123456789012", "type": "point-stream", "attributes": { "partitionId": "1bcaafc3-d4d3-43e1-ab64-89d8518d5951", "webhookUrl": "https://test-api.com/test/webhook", "pointIds": [ "88d45219-a0cf-4509-8f3d-84d15e6a9c44" ], "status": "active", "createdAt": "2025-11-17T14:30:22Z", "updatedAt": "2025-11-17T14:30:22Z" } } ``` The `id` is your stream identifier that you can use to manage this subscription later. ### List Your Stream Subscriptions To view all your registered webhooks: ```sh # List point stream subscriptions curl -H "Authorization: Bearer $TOKEN" \ "https://https://eu.buildingx.siemens.com/api/openness/streaming/v1/partitions/$PARTITION/streams?type=point-streams" ``` ### Webhook Payload Examples Once registered, your webhook endpoint will receive POST requests whenever point data changes occur. Each webhook delivery contains the updated point information in the following structure: #### Point Data Event When a point value changes, your webhook will receive a payload like this: ```json { "changed": 1, "createdAt": "2025-12-03T06:09:21.915Z", "id": "6360e1ef-18b5-457c-a488-47t4f4b60e29", "lastUpdatedAt": "2025-12-03T06:09:28.077Z", "qualityOfValue": 0, "value": "22" } ``` **Field Descriptions:** - `changed`: Indicates the change status (1 = changed, 0 = no change) - `createdAt`: Timestamp when the point was initially created - `id`: Unique identifier for the point - `lastUpdatedAt`: Timestamp of the most recent update - `qualityOfValue`: Quality indicator for the data (0 = good quality) - `value`: The current value of the point Your webhook endpoint must respond with a `2xx` HTTP status code to acknowledge successful receipt. Failed deliveries will be retried and eventually sent to the Dead Letter Queue if they continue to fail. ### Verify Webhook Signatures (Optional) If you provided a webhook secret during registration, you can verify webhook signatures to ensure authenticity: ```python import hmac import hashlib def verify_webhook_signature(payload, signature, secret): expected_signature = hmac.new( secret.encode('utf-8'), payload.encode('utf-8'), hashlib.sha256 ).hexdigest() return hmac.compare_digest(f"sha256={expected_signature}", signature) ``` ### Manage Stream Subscriptions You can get entities, add/remove entities, or delete your stream subscriptions as needed: ```sh # Get all points in a point stream curl -X GET \ -H "Authorization: Bearer $TOKEN" \ "https://https://eu.buildingx.siemens.com/api/openness/streaming/v1/partitions/$PARTITION/streams/$STREAM_ID/entities?type=point-streams" # Add or update points in a point stream curl -X PUT \ -H "Authorization: Bearer $TOKEN" \ -H "Content-Type: application/json" \ -d '{ "pointIds": [ "88d45219-a0cf-4509-8f3d-84d15e6a9c49" ] }' \ "https://https://eu.buildingx.siemens.com/api/openness/streaming/v1/partitions/$PARTITION/streams/$STREAM_ID/entities?type=point-streams" # Remove specific points from a point stream curl -X DELETE \ -H "Authorization: Bearer $TOKEN" \ -H "Content-Type: application/json" \ -d '{ "pointIds": [ "88d45219-a0cf-4509-8f3d-84d15e6a9c49" ] }' \ "https://https://eu.buildingx.siemens.com/api/openness/streaming/v1/partitions/$PARTITION/streams/$STREAM_ID/entities?type=point-stream" # Delete entire point stream subscription curl -X DELETE \ -H "Authorization: Bearer $TOKEN" \ "https://https://eu.buildingx.siemens.com/api/openness/streaming/v1/partitions/$PARTITION/streams/$STREAM_ID?type=point-stream" ``` The GET operation returns a list of entities. The entity DELETE operation returns the updated stream. The stream DELETE operations return a `204 No Content` response if successful. # Changelog The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). ## [1.1.0] - 2026-05-01 #### IMPORTANT CHANGES - **API Endpoint Update:** New domain and API path for the Sustainability API. - **Old Endpoint:** `https://api.bpcloud.siemens.com/sustainability/v1/*` - **New Endpoint:** `https://eu.buildingx.siemens.com/api/openness/sustainability/v1/*` - **Action Required:** Please update your integrations to use the new endpoint. - **Grace Period:** The old domain will continue to be supported for a grace period. A separate notification will be sent to all customers regarding the deprecation date ### [1.0.0] - 2026-03-01 #### Added - Release of Sustainability API. # Sustainability API The Sustainability API enables you to read utility data assets such as energy consumption, cost data for a customer and for the period of interest. The API is available as an add-on in combination with an active Building X Sustainability Manager subscription. ## Concepts & Glossary | Term | Description | | ----------- | ------------------------------------------------------------------------------------------------------------------- | | Consumption | It provides the consumption of a location or measured by a Meter-Equipment that is derived from its meter readings. | | Cost | Cost refers to cost generated by the energy consumption of a building. | # Quick Start Welcome to the Sustainability API Quick Start guide! This document will walk you through the essential steps to authenticate and interact with the Sustainability APIs, from account creation to making your first API requests. ## Prerequisites Before you begin, ensure you have: - Access to the Fire API portal. - Credentials for a machine user (`clientId`, `clientSecret`, and `customerId`). - A terminal environment (Linux/MacOS shell or Windows command prompt). - [curl](https://curl.se/) installed, or another HTTP client of your choice. ## Step 1: Create an Account and Machine User To access the Fire APIs, you need to create an account and register a machine user. Follow the instructions in the [Getting Started](../../dev-guide/gettingstarted.html) guide to obtain your `clientId`, `clientSecret`. You need to get your `customerID` as well. ## Step 2: Obtain a JSON Web Token (JWT) Authenticate your machine user by generating a JWT. This token will be used to authorize your API requests. ### Example: Generate a Token Set your credentials as environment variables: ```bash export CLIENT_ID= export CLIENT_SECRET= ``` **Endpoint:** `POST {{buildingxopenness.env.issUrl}}` **Body Parameters:** - `client_id` (string, required): Your machine user's client ID. - `client_secret` (string, required): Your machine user's client secret. - `audience` (string, required): The audience for the token, typically the API identifier. - `grant_type` (string, required): The OAuth2 grant type, usually `client_credentials`. **Example:** ```bash curl {{buildingxopenness.env.issUrl}} \ -H 'content-type: application/json' \ -d "{ \"client_id\":\"$CLIENT_ID\", \"client_secret\":\"$CLIENT_SECRET\", \"audience\":\"{{buildingxopenness.env.audience}}\", \"grant_type\":\"client_credentials\" }" ``` > **Note:** > > - The above example uses a Linux/MacOS shell. On Windows, use `set` instead of `export`. > - You can use any HTTP client or programming language to make these requests. #### Example Response ```json { "access_token": "eyJ0eXAiOiUSJ9.eyJpc3MiOiJdGlhbHMifQ.MJpcxLfyOt", "token_type": "Bearer", "expires_in": 86400 } ``` - The `access_token` is your JWT. - The `expires_in` value (in seconds) indicates how long the token is valid (typically 24 hours). ## Step 3: Make API Requests With your token and customer ID, you can now interact with the Sustainability API. ### Retrieve Utility Data You can fetch the utility data for a customer by performing the GET `Utility Data` operation. **Endpoint:** `GET customers/{$customerID}/assets/utility-data` **Path Parameters:** - `customerID` (string, required): The unique identifier of the customer. **Query Parameters:** - `customerName` (string, required): Name of the customer. - `pageSize` (number, optional): Specifies the response size. - `serviceType` (string, optional): Specifies the service type. **Example:** You can fetch the utility data for a customer by performing the GET `Utility Data` operation. ```bash curl -H "Authorization: Bearer $TOKEN" \ "{{buildingxopenness.env.sustainabilityApiUrl}}/customers/$CUSTOMER/assets/utility-data" ``` - The response includes a list of buildings with utility data assets for customers. - For large customers, refer to the [Pagination](../../dev-guide/index.html#pagination) section. # Changelog The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). ### [0.1.0] - 2026-05-07 #### IMPORTANT CHANGES - **API Endpoint Update:** New domain and API path for the Visitor Manager API. - **Old Endpoint:** `https://api.bpcloud.siemens.com/visitormanager/v1/*` - **New Endpoint:** `https://eu.buildingx.siemens.com/api/openness/visitormanager/v1/*` - **Action Required:** Please update your integrations to use the new endpoint. - **Grace Period:** The old domain will continue to be supported for a grace period. A separate notification will be sent to all customers regarding the deprecation date ### [0.0.1] - 2026-03-01 #### Feature - Release of Visitor Manager API # Visitor Manager API Visitor Manager API enables you to retrieve and manage visitor information within the Building X ecosystem. ## Concepts & Glossary | Term | Description | | ---------- | ------------------------------------------------------- | | Visitor | Identifies the person who is visiting the campus | | Visit | Identities the invite for which a visitor is invited to | | Assignment | Links the visitor to a visit | # Quick Start Sample quickstart for Visitor Manager API Reference Getting started with using Visitor Manager APIs involves the following steps: 1. Create an account and a machine user. 1. Create a JSON Web Token (JWT) by using the machine user credentials. 1. Make API requests using the JWT. Note In the following examples we are: 1. making use of a Linux/MacOS shell in which environmental variables are set using the `export`-command. In other environments it may be different, e.g. Windows uses the `set`-command instead. 1. using the `curl` as a client. But the API can be used in any programming language with an HTTP Client, e.g. Go, Python, NodeJS, Javascript and Java. ## Create an account and a machine user The [Getting Started](../../dev-guide/gettingstarted.html) page documents the required steps to get a hold of the `clientId`, `clientSecret` and `partitionId`. ## Create a token Use the values described in the [Authorization](../../dev-guide/howto.html)- section to construct the Create Token request. ### Example request ```bash export CLIENT_ID= export CLIENT_SECRET= curl {{env.issUrl}} \ -H 'content-type: application/json' \ -d "{ \"client_id\":\"$CLIENT_ID\", \"client_secret\":\"$CLIENT_SECRET\", \"audience\":\"{{env.audience}}\", \"grant_type\":\"client_credentials\" }" ``` To run this example yourself, set the `CLIENT_ID` and `CLIENT_SECRET` first. ### Example response ```json { "access_token": "eyJ0eXAiOiUSJ9.eyJpc3MiOiJdGlhbHMifQ.MJpcxLfyOt", "token_type": "Bearer", "expires_in": 86400 } ``` The token, or JWT (JSON Web Token), is the value of the `access_token`-property in the response. You can now use it by passing it in the `Authorization`-header of any subsequent API requests. The `expires_in`-property represents the number of seconds your token is valid, usually, the value corresponds to 24 hours. When this time has elapsed you will need to create a new token. Now you have all you need to start using the API. As a last step of preparation set the token and `partitionId` as environmental variables. ```bash export PARTITION= export TOKEN= ``` ## Make API requests Refer the [API Reference](api-reference.html) for the available endpoints and their usage. # Developer's Guide The Siemens Building X APIs are based on the [JSON:API](https://jsonapi.org/) specification. On top of JSON:API we have applied some conventions of our own. This document aims to describe those and give you other useful information to get started and operate an application based on our APIs. ## Authentication You can use your Machine User credentials to obtain an API Token using the OAuth 2.0 Client Credentials grant. The token endpoint is found at: `https://siemens-bt-015.eu.auth0.com/oauth/token` | Parameter | Value | | --------------- | ------------------------------------------------------------- | | `client_id` | (mandatory) The *client_id* you received when registering | | `client_secret` | (mandatory) The *client_secret* you received when registering | | `audience` | (mandatory) `https://horizon.siemens.com` | | `grant_type` | (mandatory) `client_credentials` | For information on how to create you own client credentials, or Machine User, check out the [Getting Started](gettingstarted.html). Refer to [Authentication](howto.html) for more hands-on guidance. ### Token Validity The token has limited validity. After expiry you may create a new token by repeating the create token flow. The JWT itself, as well as the response of the create token request, will provide information about the expiry time. In the JWT there is a `exp` claim, and in the response, there is a `expires_in` property. ## Filtering Filters are provided as query string parameters using the `filter` family style. The available filters for each list operation are described in the API Reference. #### Example Filter devices with `serialNumber` of value `12345`. ```http GET /partitions/5435b9ac-8823-9e89-804bd4710e90/devices?filter[serialNumber]=12345 ``` ## Pagination Paging functionality is provided as query string parameters using the `page` family style. The available paging strategy and specific options for each list operation are described in the API Reference. ### Controlling Page Size Page size is controlled by using the `page[limit]` param. #### Example Set the maximal number of records to receive in the response to `5`. ```text GET /partitions/5435b9ac-8823-9e89-804bd4710e90/devices?page[limit]=5 ``` ### Navigating Pages Navigation links are provided within the `links` property using different keys to represent different targets as described in the table below. | Key | Description | | ----- | ------------------------- | | self | the current page of data | | first | the first page of data | | last | the last page of data | | prev | the previous page of data | | next | the next page of data | Keys are omitted to indicate that a particular link is unavailable. When listing resources, e.g. devices, there are typically one or two links present. `self` is always present and `next` is present if there is a next page to fetch. When the `next` link is no longer in the response, then there are no more pages to fetch. ## Inclusion of related resources Building X APIs adhere to the JSON:API specification. For some operations the `include` parameter is supported. If the `include` parameter is specified for an operation it can be used with the defined values. This gives you the option to include resources that appear in the `relationships` without performing another request to the server. When used, the response will contain a top-level property `included` which contains the requested resources. In addition, there is a parameter called `field`, available for some operations. By specifying it, the client can retrieve certain fields that are not otherwise in the response. ## Request ID Every response contains a response header called `X-Request-Id`. The value of the header is a unique identifier of the transaction. Use this and share with the Development Team when reporting problems. ### Example Response ```http HTTP/1.1 400 Bad Request X-Request-Id: 2e92b826-8d7f-4a86-bf84-241c02c1b90a ``` ## JSON Error Object | Term | Description | | ------ | -------------------------------------------------------------------------------------------------------------------------------------------------------- | | id | A unique identifier for this particular occurrence of the problem. | | status | The HTTP status code applicable to this problem, expressed as a string value. | | code | An application-specific error code, expressed as a string value. | | title | A short, human-readable summary of the problem that SHOULD NOT change from occurrence to occurrence of the problem, except for purposes of localization. | ### Example Response ```json { "errors":[ { "id": "2e92b826-8d7f-4a86-bf84-241c02c1b90a", "status": "400", "code": "34-0001", "title": "Missing mandatory property 'name'" } ] } ``` ## HTTP Status Codes | Term | Description | | ---- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 200 | Standard response for successful HTTP requests. The actual response will depend on the request method used. In a GET request, the response will contain an entity corresponding to the requested resource. In a POST request, the response will contain an entity describing or containing the result of the action. | | 201 | The request has been fulfilled, resulting in the creation of a new resource. | | 202 | The request has been accepted for processing, but the processing has not been completed. The request might or might not be eventually acted upon, and may be disallowed when processing occurs. | | 204 | The server successfully processed the request and is not returning any content. | | 400 | The server cannot or will not process the request due to an apparent client error (e.g., malformed request syntax, size too large, invalid request message framing, or deceptive request routing). | | 401 | Similar to 403 Forbidden, but specifically for use when authentication is required and has failed or has not yet been provided. | | 403 | The request was valid, but the server is refusing action. The user might not have the necessary permissions for a resource, or may need an account of some sort. This code is also typically used if the request provided authentication via the WWW-Authenticate header field, but the server did not accept that authentication. | | 404 | The requested resource could not be found but may be available in the future. Subsequent requests by the client are permissible. | | 405 | A request method is not supported for the requested resource; for example, a GET request on a form that requires data to be presented via POST, or a PUT request on a read-only resource. | | 406 | The requested resource is capable of generating only content not acceptable according to the Accept headers sent in the request. | | 409 | Indicates that the request could not be processed because of conflict in the current state of the resource, such as an edit conflict between multiple simultaneous updates. | | 429 | The user has sent too many requests in a given amount of time. Intended for use with rate-limiting schemes. | | 500 | A generic error message, given when an unexpected condition was encountered and no more specific message is suitable. | | 503 | The server cannot handle the request (because it is overloaded or down for maintenance). Generally, this is a temporary state. | | 504 | The server was acting as a gateway or proxy and did not receive a timely response from the upstream server. | ## Standards & Formats | Term | Description | | ----------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | [RFC 3339 Date and Time on the Internet: Timestamps](https://tools.ietf.org/html/rfc3339) | The chosen way to represent time and dates. | | [The OAuth 2.0 Authorization Framework](https://datatracker.ietf.org/doc/html/rfc6749) | OAuth 2.0 for authorization and `client_credentials` grant type. | | [WGS84](https://en.wikipedia.org/wiki/World_Geodetic_System) | The representation of geographical coordinates. | | [Project Haystack](https://project-haystack.org/) | Project Haystack is an open source initiative to streamline working with data from the Internet of Things in the domain of Building Automation. | | [RFC7946 The GeoJSON Format](https://datatracker.ietf.org/doc/html/rfc7946) | GeoJSON is a geospatial data interchange format based on JavaScript Object Notation (JSON). | | [JSON:API](https://jsonapi.org/) | JSON:API is a specification for how a client should request that resources be fetched or modified, and how a server should respond to those requests. JSON:API is designed to minimize both the number of requests and the amount of data transmitted between clients and servers. This efficiency is achieved without compromising readability, flexibility, or discoverability. | ## Policies ### Rate Limitation Each client may make 10 API request per second. If you have additional requirements please contact us. Exceeding the rate limit will result in a `429` error. ### Compatibility Policy The APIs will evolve over time by the introduction of new properties and endpoints. Prepare your parsers for new properties, so that they are allowed in order not to break the client. ### Backwards Compatibility As much as possible, we strive not to change the behavior of existing properties. However, the behavior of an API may change without warning if the existing behavior is incorrect or constitutes a security vulnerability. ### Deprecation We reserve the right to deprecate APIs in full or in part. Usually this is only done after introducing improved ways to achieve the same goals. When deprecating an API, we will notify all registered users at least 3 months in advance. # Getting Started The Siemens Building X APIs enable you to retrieve up-to-date information from your building. To use our APIs, depending on whether you already have a customer account or not, you can either go directly to the [Building X API Manager](#api-manager), or try out the APIs using our [sandbox environment](#sandbox-environment) and Developer Portal. Are you interested in obtaining a customer account or connecting your building? Contact Siemens Regional Support. ## API Manager If you already have a customer account and a connected building: 1. Log in to [API Manager](https://eu.buildingx.siemens.com/openness/). 1. Go to the **Machine Users** page and create a new machine user. Ensure that your machine user has the correct user groups assigned. Note down your `clientId` and `clientSecret`. 1. Go to the **API documentation** page, select your machine user and partition, then generate the token by providing the client secret key. 1. You are ready to call the APIs! Select the APIs of interest from the navigation and try out your first calls. Note You need to have the role `Company Administrator` to perform the above steps. If you cannot see the **Machine Users** menu, contact the administrator in your organization to help you perform the steps. ## Sandbox environment If you don't have a customer account, you can still try out our APIs: 1. [Request access via the developer dashboard](https://dashboard.accounts.developer.siemens.com/request-access/1000) to our sandbox environment. The credentials are granted immediately and you can inspect them on the [My Credentials](https://dashboard.accounts.developer.siemens.com/credentials) page. 1. Once you obtain the credentials, refer to the [Authentication](howto.html) page to learn how to obtain an authorization token. 1. To start calling our APIs directly from the Developer Portal, go to our [try-out page](try-out.html). Click the "Authorize" button to enter your token and start testing the APIs. Note Currently, the Fire and LifecycleTwin APIs are not part of the sandbox environment. Please use [Contact Us](../contact.html) to request access. # Authentication This chapter aims to provide hands-on examples on how to authenticate by creation of an authorization token and making a sample call. ### Example Request Use the sample below and fill your `client_id` and `client_secret` to request an authorization token: ```bash export CLIENT_ID= export CLIENT_SECRET= curl https://siemens-bt-015.eu.auth0.com/oauth/token \ -H 'content-type: application/json' \ -d "{ \"client_id\":\"$CLIENT_ID\", \"client_secret\":\"$CLIENT_SECRET\", \"audience\":\"https://horizon.siemens.com\", \"grant_type\":\"client_credentials\" }" ``` ### Example Response ```json { "access_token": "eyJ0eXAiOiUSJ9.eyJpc3MiOiJdGlhbHMifQ.MJpcxLfyOt", "token_type": "Bearer", "expires_in": 86400 } ``` The token, or JWT (JSON Web Token), is the value of the `access_token` property in the response. You can now use it by passing it in the `Authorization` header of any subsequent API requests. The `expires_in` property represents the number of seconds your token is valid for, usually the value corresponds to 24 hours. When this time has elapsed you will need to create a new token. For example, when using the **Operations API** to list **Devices**: ```bash export TOKEN=eyJ0eXAiOiUSJ9.eyJpc3MiOiJdGlhbHMifQ.MJpcxLfyOt export PARTITION=1bcaafc3-d4d3-43e1-ab64-89d8518d5951 curl -H "Authorization: Bearer $TOKEN" "https://eu.buildingx.siemens.com/api/openness/operations/partitions/$PARTITION/devices" ``` Note Your token is valid for a limited time and your application will need to refresh it at regular intervals. Warning Make sure to store your token in a safe place, never share it or store it in source control. Anyone with access to your token can access the system on your behalf. # Try Out Info In order to try out APIs, please see [Authentication](howto.html) to obtain the token and authorize via the button. You can obtain the partition ID from the Accounts API. # Building X Connector for Mendix The Building X Connector for Mendix simplifies the creation of new business value using LowCode powered by Mendix and the Building X platform. This Connector provides a domain model encompassing all core entities exposed by Building X. It also offers actions for retrieving, creating, updating, and deleting these entities, which become available in the Mendix toolbox upon installing the Connector. This document will guide you through the structure of the Connector, enabling you to build innovative applications. ## Firewall Settings for Building X Cloud The following tables describe the Building X cloud endpoints used by the Siemens ID login service, applications, and the machine user. All data communication via the Internet is encrypted using Transport Layer Security (TLS) 1.2 or higher. Building X uses modern cloud principles (such as content delivery networks) to achieve high availability and scalability. The cloud endpoint DNS names can be resolved to a wide range of IP addresses based on caller context and backend state. To access the Building X APIs, connections must be established to all cloud endpoints listed in Building X platform, Building X Data Onboarding, and in the following table. ### Service Endpoints API Manager is a service that provides the capability to manage access to APIs in Building X. | Service | Cloud Endpoint | TCP-Port | | ----------------------- | ------------------------------------------------------------------------------------------------- | -------- | | API Manager application | | 443 | | API Entry | [https://eu.buildingx.siemens.com/api/openness/](https://eu.buildingx.siemens.com/api/openness/*) | 443 | ## Importing the Building X Connector This guide will walk you through the steps to import the 'Building X Connector for Mendix' from the Mendix Marketplace into your Mendix application. **Prerequisites**: - A Mendix account - Access to Mendix Studio Pro - An existing Mendix app or a new app created Complete the following steps to import the Building X Connector for Mendix. 1. **Open Your Mendix App**: 1. Launch Mendix Studio Pro. 1. Open the Mendix app into which you want to import the 'Building X Connector for Mendix'. 1. **Access the Mendix Marketplace**: 1. In the top menu, select `App` and then select `Marketplace` from the dropdown list.\ The Mendix Marketplace opens within the Studio Pro interface. 1. **Search for the Connector**: 1. In the Marketplace search bar, type 'Building X Connector for Mendix'. 1. Press `Enter` to search. 1. **Select the Connector**: 1. From the search results, find the 'Building X Connector for Mendix'. 1. Select the Connector to view its details. 1. **Import the Connector**: 1. On the Connector details page, select `Download`.\ Once the download is complete, you will see a prompt to add the Connector to your app. 1. Select `Add` to proceed. 1. **Configure the Connector**: 1. After adding the Connector, you must configure it to work with your app.\ Follow the documentation provided with the Connector for specific configuration steps. This typically involves setting up API keys, endpoints, and other necessary settings. 1. **Use the Connector in Your App**:\ With the 'Building X Connector for Mendix' imported and configured, you can now start using it in your Mendix app. 1. Navigate to the relevant part of your app where you want to utilize the Connector and implement it as needed. 1. **Test Your App**: 1. Once the Connector is integrated, thoroughly test your app to ensure it works as expected. 1. **Check for Issues**: 1. Check for any issues or errors and consult the Connector’s documentation for troubleshooting tips. ## Concepts This chapter outlines the key architectural concepts for the **Building X Connector for Mendix**. ### Entity Model The Connector provides a comprehensive entity model that includes entities that can be retrieved from or stored with Building X. This rich entity model streamlines application development, making the development process easier and more efficient. #### Non-persistence The entity model exposed by the Connector is intentionally non-persistent to reduce potential concurrency issues when executing multiple requests simultaneously. When integrating the Connector, the application has the flexibility to customize the Connector's domain model into persistent entity classes within its architecture and data storage model. This customization allows the application to tailor the data model to its unique use cases and storage requirements, efficiently optimizing performance and resource utilization. ### Authentication and Token Management The Connector automatically handles token management, ensuring seamless authentication for users. It only requests a new token when the current token expires, streamlining the authentication process and improving the user experience. The BX Authenticate action takes a machine user ID and secret as input. It returns a token that is valid for at least another 5 minutes, providing secure access to the underlying Building X APIs. ### Associations In Mendix, it's best to design pages from a domain model once the relationships between entities are established. This allows Mendix to navigate between entities and populate UI elements. However, this approach may retrieve too many Referenced Entities when loading a single entity from a data store or service. When using persistent entities, Mendix addresses this with the *Retrieve by Association* option in the `Retrieve Objects` action, which loads Referenced Entities as needed. By default, Referenced Entities are not loaded until requested, streamlining data retrieval. The Connector handles entity retrieval and association differently due to the fact that child entities cannot always be loaded directly from their parent entity when querying the Building X platform. When entities are loaded from Building X into a microflow, the associated entities become linked with the retrieved entity. If the associated entity is missing, the Connector creates a new instance of the referenced entity, only initializing its `Identifier` attribute. Once loaded, the entity instance integrates into the microflow context, ensuring smooth data flow and operation. Initialize the referenced entity before or after the action requiring it. The Connector manages Referenced Entities in the same microflow context, promoting efficient data reuse and integration. This approach not only enhances performance and productivity but also fosters seamless navigation across entities within microflows and pages. In a microflow, you can either create the referenced entity from Building X or create it from your local data store. This ensures that all associated entities are initialized correctly, which is important for navigating entity relationships in Mendix. The `BX Track Entities` action adds entities to a list of tracked entities. This list helps manage associations when an entity with associations is loaded by a retrieved entity from the Platform. The tracked and thus loaded entity references are used to associate entities with each other. If an associated entity is not already tracked, a new instance of the entity is created, with only the Identifier field set. The `BX Fetch Referenced Entities` action retrieves all entities that are referenced by the entities you have already retrieved, ensuring that all necessary related data is included in the microflow. When used, this action identifies all entities that are referenced by the currently retrieved entities in the context of the current microflow. For each referenced entity, only the Partition ID and Identifier are initialized, while all other fields retain their default values. Once this action is executed, all uninitialized entities are retrieved, ensuring that all fields of entities used in the microflow are populated. When fetching partially initialized entities that contain new references to other entities, the referenced entities remain uninitialized. An additional call to the action is required to fully initialize those entities. Info The `Fetch Referenced Entities` action uses multi-threading to send multiple requests to Building X to retrieve entities simultaneously. By default, a maximum of 32 threads (and thus 32 simultaneous requests) are used to retrieve referred entities as efficiently as possible. Info For every entity to be initialized, a request is sent to Building X. Therefore, it may be more efficient to load the desired entities in a batch using a corresponding list action, such as `Get Location List`. When restoring entities from the local datastore, prepopulate a `List` object and supply it to the action requiring initialized entities. Mendix transmits entities from the client to the server using the `List` object as a parameter value for the action. This allows the retrieval action to reuse the populated entities. #### Example 1 Consider a scenario where you want to initialize a list of `EquipmentEntity` entities from Building X. Once the `Get Equipments` action is executed, a list of `equipment` is available. The association to `equipment type` is made, but because the `EquipmentType` entity itself is not loaded, only the `Identifier` attribute is set. #### Example 2 Consider a scenario where you want to initialize a list of `EquipmentEntity` entities from Building X where the `Equipment Type` can be selected from a dropdown list in your application. In this scenario, you need to execute the `Get Equipment Types` and `Get Equipments` actions. Because entity instances are shared within the context of a microflow, it does not matter if the `Get Equipment Types` action is performed before or after the `Get Equipments` action. In either order, the `equipment` and `equipment type` entities are populated and associated. If the `Get Equipments` action precedes the `Get Equipment Types` action, the `Equipment` entities are linked to an `Equipment Type`, but only the `Identifier` of this entity is established. Subsequently, upon executing the `Get Equipment Types` action, all attributes of the `Equipment Type` entities are populated. If the `Get Equipments` action precedes the `Get Equipment Types` action, the `Equipment` entities are linked to an `Equipment Type`, but only the `Identifier` of this entity is established. Subsequently, upon executing the `Get Equipment Types` action, all attributes of the `Equipment Type` entities are populated. #### Example 3 Consider a scenario where you want to initialize a list of `EquipmentEntity` entities without retrieving the already loaded `EquipmentTypeEntity` entities stored in the local database. In this situation, you can instantiate `EquipmentTypeEntity` from the local database and include them in the list while retrieving `EquipmentEntity` entities using the `associatedEquipmentTypes` parameter. This strategy of using a pre-populated list not only encourages reuse of equipment type references but also improves efficiency within your application. ### Identity Handling In Building X, the management of identifiers is crucial for creating, updating, and retrieving various entities within the platform. For most entities in Building X, the platform automatically generates unique identifiers. This simplifies the process for developers, as they do not need to manually create or manage these identifiers. When an entity is created, the platform generates a unique identifier and includes it in the response payload after the action is executed. This ensures that each entity can be uniquely and reliably identified within the platform. While the platform generally handles identifier generation, there is an exception for `LocationEntity` entities. Developers have the option to provide their own identifiers when creating location entities. These client-generated identifiers must be in the Universally Unique Identifier (UUID) version 4 format. The pattern for these UUIDs follows the regular expression: `^[A-Z]?[a-fA-F0-9]{8}-[a-fA-F0-9]{4}-[a-fA-F0-9]{4}-[a-fA-F0-9]{4}-[a-fA-F0-9]{12}$`. ### Tags Handling The `Tag` entity is designed to facilitate the definition and management of tag names and values for entities that support tagging, such as `LocationEntity` or `PointEntity`. Tag values can be either the boolean marker "true" or an array of strings. Before using tags, the `BuildingX_Connector.Tag` relationship must refer to a valid instance of a domain entity that extends the functionality of the `EntityWithTags` entity, such as `LocationEntity` or `PointEntity`. Additionally, tag names must not be null or empty. Tags with null or empty names or values are excluded without generating an error. When a tag's value is empty or equals the string `true`, it is stored as a marker. Otherwise, the string value is saved in an array of strings. If a tag with the same name is created multiple times, all unique values are consolidated into the array of strings, excluding any empty strings. Tags are organized by their names, ensuring that values are unique within each tag. After processing, all tag names are trimmed of leading and trailing whitespace, while the tag values remain unchanged. ## Error Handling When executing an action within Building X, exceptions may occur in the following scenarios: - Invalid input parameters are detected for the action. - Validation for creating or updating an entity fails. - The HTTP error code returned by the underlying endpoint does not indicate success (200 OK). In such cases, it's crucial to utilize Mendix's default error handling functionality to gracefully manage these failed requests. For instance, configuring a microflow within Mendix can effectively address the exception by logging the error, notifying administrators, or presenting a user-friendly error message to the end user. This approach ensures a smoother user experience and facilitates efficient troubleshooting and issue resolution. ## Minimal Microflow The minimal microflow consists of the following steps: 1. **Authenticate Machine User and Request Token**: Call the `BX Authenticate` action to obtain a token. 1. **Building X Action**: Provide the obtained token as input to the Get, Create, Update, or Delete action. For example the `Get Devices` or `Create Equipement` action. Info It is recommended to retrieve these configurations from a persistent settings entity associated with the current user to securely store and manage this data. Alternatively, constants can be utilized with different Mendix configuration profiles to identify machine user credentials. ## Actions This chapter describes the action naming and parameter conventions, as well as a comprehensive list of all actions provided by the Connector. ### Action Naming The Connector follows a consistent approach for managing entities. The following actions are available for every entity: | Action | Description | | ----------------- | ---------------------------------------------------------------------------------------------- | | Get `` | Retrieves all entities that match the given filter parameters. | | Get `` | Fetches a single entity by its identifier. | | Add `` | Add a new entity from the newly created non-persistent entity. The created entity is returned. | | Update `` | Updates the given non-persistent entity and returns the updated entity. | | Delete `` | Deletes the given entity. | Note Data retrieval is applicable to all entities with the corresponding API subscription and access rights. Some entities support only reading. ### Parameter Categories When performing actions with the Building X Connector, various parameter categories are utilized to configure the requests: | Category | Description | | -------------- | --------------------------------------------------------------------------------------- | | Authorization | Token required to access the underlying API endpoint. | | Identification | Parameters used to identify the entity, such as the company or partition and entity ID. | | Filters | Filters applied to select a subset of the requested entities. | | Parameters | Additional parameters used to include supplementary data or configure the action. | | Associations | List of already-loaded entities to reuse as associated entities. | These categories help structure the inputs required for interacting with the Building X Connector efficiently. ## Action Overview This table provides an overview of the various actions available within the BuildingX Connector. Each action is listed with its associated name, input parameters, and output. These actions facilitate interactions with BuildingX's services, enabling functionalities such as authorization, fetching and updating entities, managing equipment, and handling energy consumption data. Convert the 'Endpoint' column to hyperlinks, use the second last part, before api-reference, of the url as the hyperlink text. Export MkDocs style and include all rows until 'bx_Device_Create'. Convert the 'Endpoint' column to hyperlinks, use the second last part, before api-reference, of the url as the hyperlink text. Export MkDocs style and include all rows until 'bx_Device_Create'. ### General The Connector provides several actions to ease development effort, offering streamlined micflows for tasks such as data retrieval and updates. Each action includes clear inputs, outputs, and API endpoints, ensuring efficient integration and enhancing functionality for developers. | Name | Action | Input Parameters | Output | Description | | -------------------------------- | -------------------------- | ------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **BX Authorize** | bx_Authorize | MachineUserId (String) MachineUserSecret (String) | Token | Authorizes a machine user and retrieves a token. | | **BX Get Location Children** | bx_Location_Children_Get | token (BuildingX_Connector.Token) parentLocation (BuildingX_Connector.LocationEntity) recursive (Boolean) includeSelf (Boolean) | List of BuildingX_Connector.LocationEntity | This action retrieves child locations for a specified parent location. It allows for the inclusion of the parent location and recursive retrieval of all child locations. If child locations are not loaded, they are fetched from the Platform using the provided token and parent location details. | | **BX Fetch Referenced Entities** | bx_FetchReferencedEntities | token (BuildingX_Connector.Token) | Nothing | Retrieves all entities that are used as references by the entities retrieved from the Platform. | | **BX Track Entities** | bx_TrackEntities | List of EntityWithIdentifier | Nothing | Adds entities to a list of tracked entities. This list is used to manage associations when an entity, which has associations, is loaded by a retrieved entity from the Platform. | ### Building Twin The Connector provides several actions to ease development effort, including those specifically aimed at facilitating the creation and management of digital twins for buildings. These actions enable developers to efficiently retrieve, update, and synchronize building data, enhancing the accuracy and responsiveness of applications that rely on digital representations of real-world buildings. Notably, Building Twin does not include Life Cycle Twin functionalities. | Name | Action | Input Parameters | Output | API endpoint | Roles | Description | | ---------------------- | ------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------- | --------------------------------------------------------------------------------------------------------------------- | ----------------------- | ----------------------------------- | | **Add Address** | bx_Address_Create | token (Token) partitionId (String) address (AddressEntity) | AddressEntity | [Building Structure API](https://developer.siemens.com/building-x-openness/api/building-structure/api-reference.html) | Structure API ReadWrite | Creates a new address. | | **Add Location** | bx_Location_Create | token (Token) partitionId (String) location (LocationEntity) includeReferredEntities (Boolean) | LocationEntity | [Building Structure API](https://developer.siemens.com/building-x-openness/api/building-structure/api-reference.html) | Structure API ReadWrite | Creates a new location. | | **Delete Address** | bx_Address_Delete | token (Token) address (AddressEntity) | Nothing | [Building Structure API](https://developer.siemens.com/building-x-openness/api/building-structure/api-reference.html) | Structure API ReadWrite | Deletes an address. | | **Delete Location** | bx_Location_Delete | token (Token) partitionId (String) location (LocationEntity) | Nothing | [Building Structure API](https://developer.siemens.com/building-x-openness/api/building-structure/api-reference.html) | Structure API ReadWrite | Deletes a location. | | **Get Address** | bx_Address_Get | token (Token) partitionId (String) | LocationEntity | [Building Structure API](https://developer.siemens.com/building-x-openness/api/building-structure/api-reference.html) | Structure API Read | Retrieves a specific address. | | **Get Addresses** | bx_Addresses_Get | token (Token) partitionId (String) associatedCountries (List of CountryEntity) | List of AddressEntity | [Building Structure API](https://developer.siemens.com/building-x-openness/api/building-structure/api-reference.html) | Structure API Read | Retrieves a list of addresses. | | **Get Building Types** | bx_Locations_BuildingTypes_Get | token (Token) partitionId (String) | List of BuildingTypeEntity | [Building Structure API](https://developer.siemens.com/building-x-openness/api/building-structure/api-reference.html) | Structure API Read | Retrieves a list of building types. | | **Get Countries** | bx_Countries_Get | token (Token) partitionId (String) | List of CountryEntity | [Building Structure API](https://developer.siemens.com/building-x-openness/api/building-structure/api-reference.html) | Structure API Read | Retrieves a list of countries. | | **Get Location** | bx_Location_Get | token (Token) partitionId (String) locationId (String) | LocationEntity | [Building Structure API](https://developer.siemens.com/building-x-openness/api/building-structure/api-reference.html) | Structure API Read | Retrieves a specific location. | | **Get Locations** | bx_Locations_Get | token (Token) partitionId (String) includeReferredEntities (Boolean) includeHistoricalData (Boolean) associatedEquipments (List of EquipmentEntity) | List of LocationEntity | [Building Structure API](https://developer.siemens.com/building-x-openness/api/building-structure/api-reference.html) | Structure API Read | Retrieves a list of locations. | | **Update Address** | bx_Address_Update | token (Token) partitionId (String) address (AddressEntity) | AddressEntity | [Building Structure API](https://developer.siemens.com/building-x-openness/api/building-structure/api-reference.html) | Structure API ReadWrite | Updates an address. | | **Update Location** | bx_Location_Update | token (Token) partitionId (String) location (LocationEntity) includeReferredEntities (Boolean) | LocationEntity | [Building Structure API](https://developer.siemens.com/building-x-openness/api/building-structure/api-reference.html) | Structure API ReadWrite | Updates a location. | ### Building Automation The Connector provides several actions to ease development effort, including those tailored for building automation. These actions enable developers to effectively control and monitor building systems such as HVAC, lighting, and security. By utilizing these micflows, developers can streamline the integration of automation functionalities into their applications, ensuring buildings operate efficiently and securely. | Name | Action | Input Parameters | Output | API endpoint | Roles | Description | | --------------------------------- | ------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | ------------------------ | ---------------------------------------------------------------------------------------------- | | **Add Alarm Configuration** | bx_AlarmConfiguration_Create | token (Token) alarmConfiguration (AlarmConfigurationEntity) | AlarmConfigurationEntity | [Building Operations API](https://developer.siemens.com/building-x-openness/api/building-operations/api-reference.html) | Operations API ReadWrite | Creates a new alarm configuration. | | **Add Device** | bx_Device_Create | token (Token) partitionId (String) device (DeviceEntity) | DeviceEntity | [Building Operations API](https://developer.siemens.com/building-x-openness/api/building-operations/api-reference.html) | Operations API ReadWrite | Creates a new device. | | **Add Equipment** | bx_Equipment_Create | token (Token) equipment (EquipmentEntity) | EquipmentEntity | [Building Structure API](https://developer.siemens.com/building-x-openness/api/building-structure/api-reference.html) | Structure API ReadWrite | Creates new equipment. | | **Add Point** | bx_Point_Create | token (Token) point (PointEntity) | PointEntity | [Building Structure API](https://developer.siemens.com/building-x-openness/api/building-structure/api-reference.html) | Structure API ReadWrite | Creates a new point. | | **Update Alarm Configuration** | bx_AlarmConfiguration_Update | token (Token) alarmConfiguration (AlarmConfigurationEntity) | AlarmConfigurationEntity | [Building Operations API](https://developer.siemens.com/building-x-openness/api/building-operations/api-reference.html) | Operations API ReadWrite | Updates an alarm configuration. | | **Update Device** | bx_Device_Update | token (Token) partitionId (String) device (DeviceEntity) updateLocation (Boolean) | DeviceEntity | [Building Operations API](https://developer.siemens.com/building-x-openness/api/building-operations/api-reference.html) | Operations API ReadWrite | Updates a device. | | **Update Equipment** | bx_Equipment_Update | token (Token) equipment (EquipmentEntity) updateLocation (Boolean) | EquipmentEntity | [Building Structure API](https://developer.siemens.com/building-x-openness/api/building-structure/api-reference.html) | Structure API ReadWrite | Updates equipment. | | **Update Point in Point Group** | bx_PointGroup_HasPoints | token (Token) partitionId (String) pointGroupId (String) pointId (String) operation (Enumeration PointGroup_Point_Operation) | Nothing | [Building Structure API](https://developer.siemens.com/building-x-openness/api/building-structure/api-reference.html) | Structure API Read | Checks if a point group has specific points. | | **Delete Alarm Configuration** | bx_AlarmConfiguration_Delete | token (Token) alarmConfiguration (AlarmConfigurationEntity) | Nothing | [Building Operations API](https://developer.siemens.com/building-x-openness/api/building-operations/api-reference.html) | Operations API ReadWrite | Deletes an alarm configuration. | | **Delete Device** | bx_Device_Delete | token (Token) device (DeviceEntity) | Nothing | [Building Operations API](https://developer.siemens.com/building-x-openness/api/building-operations/api-reference.html) | Operations API ReadWrite | Device deletion is currently not supported. This feature will be available in a future update. | | **Delete Equipment** | bx_Equipment_Delete | token (Token) equipment (EquipmentEntity) | Nothing | [Building Structure API](https://developer.siemens.com/building-x-openness/api/building-structure/api-reference.html) | Structure API ReadWrite | Deletes equipment. | | **Delete Point** | bx_Point_Delete | token (Token) point (PointEntity) | Nothing | [Building Structure API](https://developer.siemens.com/building-x-openness/api/building-structure/api-reference.html) | Structure API ReadWrite | Deletes a point. | | **Get Last Values in PointGroup** | bx_PointGroup_GetLastValues | token (Token) partitionId (String) pointGroupId (String) | List of BuildingX_Connector.PointValueEntity | [Building Structure API](https://developer.siemens.com/building-x-openness/api/building-structure/api-reference.html) | Structure API Read | Retrieves the latest values for all points within the specified point group. | | **Get Point** | bx_Point_Get | token (Token) partitionId (String) pointId (String) | PointEntity | [Building Structure API](https://developer.siemens.com/building-x-openness/api/building-structure/api-reference.html) | Structure API Read | Retrieves a specific point. | | **Get Points** | bx_Points_Get | token (Token) partitionId (String) includeReferredEntities (Boolean) includeHistoricalData (Boolean) associatedLocations (List of LocationEntity) associatedEquipments (List of EquipmentEntity) associatedPointTypes (List of PointTypeEntity) | List of PointEntity | [Building Structure API](https://developer.siemens.com/building-x-openness/api/building-structure/api-reference.html) | Structure API Read | Retrieves a list of points. | | **Get Point Values** | bx_PointValues_Get | token (BuildingX_Connector.Token) partitionId (String) pointId (String) filterFrom (Date and time) filterTo (Date and time) | List of BuildingX_Connector.PointValueEntity | [Building Structure API](https://developer.siemens.com/building-x-openness/api/building-structure/api-reference.html) | Structure API Read | Retrieves historical data for a specific point. | | **Get Alarm Configurations** | bx_AlarmConfigurations_Get | token (Token) partitionId (String) pointId (String) | List of AlarmConfigurationEntity | [Building Operations API](https://developer.siemens.com/building-x-openness/api/building-operations/api-reference.html) | Operations API Read | Retrieves a list of alarm configurations. | | **Get Device** | bx_Device_Get | token (Token) partitionId (String) deviceId (String) | DeviceEntity | [Building Operations API](https://developer.siemens.com/building-x-openness/api/building-operations/api-reference.html) | Operations API Read | Retrieves a specific device. | | **Get Devices** | bx_Devices_Get | token (Token) partitionId (String) includeDeviceInfo (Boolean) includeConnectivity (Boolean) filterByGatewayId (String) filterByLocationId (String) associatedLocations (List of LocationEntity) associatedTimeZones (List of TimeZoneEntity) | List of DeviceEntity | [Building Operations API](https://developer.siemens.com/building-x-openness/api/building-operations/api-reference.html) | Operations API Read | Retrieves a list of devices. | | **Get Device Events** | bx_DeviceEvents_Get | token (Token) partitionId (String) deviceId (String) associatedDevices (List of DeviceEntity) | List of DeviceEventEntity | [Building Operations API](https://developer.siemens.com/building-x-openness/api/building-operations/api-reference.html) | Operations API Read | Retrieves a list of device events. | | **Get Device Events Transitions** | bx_DeviceEventsTransitions_Get | token (Token) partitionId (String) deviceId (String) eventId (String) associatedDevices (List of DeviceEntity) | List of DeviceEventEntity | [Building Operations API](https://developer.siemens.com/building-x-openness/api/building-operations/api-reference.html) | Operations API Read | Retrieves a list of device event transitions. | | **Get Equipment** | bx_Equipment_Get | token (Token) equipmentId (String) | EquipmentEntity | [Building Structure API](https://developer.siemens.com/building-x-openness/api/building-structure/api-reference.html) | Structure API Read | Retrieves a specific piece of equipment. | | **Get Equipments** | bx_Equipments_Get | token (Token) partitionId (String) includeHistoricalData (Boolean) associatedLocations (List of LocationEntity) associatedPoints (List of PointEntity) includeReferredEntities (Boolean) | List of EquipmentEntity | [Building Structure API](https://developer.siemens.com/building-x-openness/api/building-structure/api-reference.html) | Structure API Read | Retrieves a list of equipment. | | **Command Point** | bx_Point_Command | token (Token) partitionId (String) pointId (String) value (Decimal) | Boolean | [Building Operations API](https://developer.siemens.com/building-x-openness/api/building-operations/api-reference.html) | Operations API ReadWrite | Commands a specific point with a given value. | ### Energy The Connector provides several actions to ease development effort, including those designed for building energy management. These actions empower developers to monitor and optimize energy consumption, track costs, and analyze emissions. By leveraging these micflows, developers can seamlessly integrate energy management features into their applications, enhancing sustainability and operational efficiency. | Name | Action | Input Parameters | Output | API endpoint | Roles | Description | | ------------------------------------------ | -------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------ | ---------------------------------------------------------------------------------------------------------- | --------------- | -------------------------------------------------------- | | **Get Energy Medium Consumptions** | bx_EnergyMediumConsumptions_Get | token (Token) partitionId (String) filterFrom (Date and time) filterTo (Date and time) filterByMediumType (Enumeration EnergyMediumConsumption_MediumTypes) associatedEquipments (List of EquipmentEntity) associatedMediumConsumptions (List of EnergyMediumConsumptionEntity) | List of EnergyMediumConsumptionEntity | [Building Energy API](https://developer.siemens.com/building-x-openness/api/energy-api/api-reference.html) | Energy API Read | Retrieves a list of energy medium consumptions. | | **Get Energy Medium Consumption Cost** | bx_EnergyMediumConsumptionCost_Get | token (Token) partitionId (String) mediumConsumptionId (String) from (Date and time) to (Date and time) unit (Enumeration EnergyMediumConsumption_Currencies) interval (Enumeration EnergyMediumConsumption_Intervals) associatedMediumConsumptions (List of EnergyMediumConsumptionEntity) | List of EnergyMediumConsumptionValueEntity | [Building Energy API](https://developer.siemens.com/building-x-openness/api/energy-api/api-reference.html) | Energy API Read | Retrieves a list of energy medium consumption costs. | | **Get Energy Medium Consumption Emission** | bx_EnergyMediumConsumptionEmission_Get | token (Token) partitionId (String) mediumConsumptionId (String) from (Date and time) to (Date and time) unit (Enumeration EnergyMediumConsumption_EmissionsUnits) interval (Enumeration EnergyMediumConsumption_Intervals) associatedMediumConsumptions (List of EnergyMediumConsumptionEntity) | List of EnergyMediumConsumptionValueEntity | [Building Energy API](https://developer.siemens.com/building-x-openness/api/energy-api/api-reference.html) | Energy API Read | Retrieves a list of energy medium consumption emissions. | | **Get Energy Medium Consumption Usage** | bx_EnergyMediumConsumptionUsage_Get | token (Token) partitionId (String) mediumConsumptionId (String) from (Date and time) to (Date and time) unit (Enumeration EnergyMediumConsumption_UsageUnits) interval (Enumeration EnergyMediumConsumption_Intervals) associatedMediumConsumptions (List of EnergyMediumConsumptionEntity) | List of EnergyMediumConsumptionValueEntity | [Building Energy API](https://developer.siemens.com/building-x-openness/api/energy-api/api-reference.html) | Energy API Read | Retrieves a list of energy medium consumption usage. | ### Security | Name | Action | Input Parameters | Output | API endpoint | Roles | Description | | --------------------------------- | -------------------------- | ------------------------------------------------------------------------ | ----------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ | ----------------- | ------------------------------------------------------- | | **Get Security Assignments** | bx_SecurityAssignments_Get | token (BuildingX_Connector.Token) partitionId (String) identity (String) | List of BuildingX_Connector.PrivilegeEntity | [Building Operations API](https://developer.siemens.com/building-x-openness/api/security-api/piam-v1/api-reference.html) | Security API Read | Retrieves security assignments for a specific identity. | | **Get Security Alarm and Events** | bx_SecurityEvents_Get | token (BuildingX_Connector.Token) partitionId (String) | List of BuildingX_Connector.SecurityEventEntity | [Security Alarms and Events API](https://developer.siemens.com/building-x-openness/api/activities-v1/api-reference.html) | Security API Read | Retrieves security events for a specific partition. | | **Get Security Identities** | bx_SecurityIdentities_Get | token (BuildingX_Connector.Token) partitionId (String) | List of BuildingX_Connector.IdentityEntity | [Identities and Privileges API](https://developer.siemens.com/building-x-openness/api/security-api/piam-v1/api-reference.html) | Security API Read | Retrieves security identities for a specific partition. | | **Get Security Privileges** | bx_SecurityPrivileges_Get | token (BuildingX_Connector.Token) partitionId (String) | List of BuildingX_Connector.PrivilegeEntity | [Identities and Privileges API](https://developer.siemens.com/building-x-openness/api/security-api/piam-v1/api-reference.html) | Security API Read | Retrieves security privileges for a specific partition. | ## Building X Subscriptions The Building X Connector leverages various Building X APIs to facilitate its operations. To explore and review the available API products that underpin these functionalities, please visit the [Xcelerator Marketplace](https://xcelerator.siemens.com/global/en/all-offerings.html?selectedFilters=1-b371c5c2-1633-4bf4-a171-66241aa8a11b%3B2-8e07596f-2368-4d1b-ba11-9cf46160a42e%3B3-f30e97f8-089e-4847-9e97-9a66e8dffc68). ## Use Cases Building X integrated with Mendix offers powerful capabilities for developing and maintaining applications with significantly reduced effort compared to high-code alternatives ### How to Get Devices Behind a Gateway In scenarios where multiple devices are connected through a gateway device, the gateway acts as a mediator, relaying information between the external network and the internal devices. The gateway uses network scanning, device discovery protocols, and its management interface to list all connected devices. This process is crucial for monitoring, controlling, and updating devices seamlessly. To retrieve device information from devices connected to this gateway, additional actions are required. See the `DS_GetDevicesBehindGateway` microflow from the *Building X Connector for Mendix* Sample App for details. ### How to Get the Last Value Per Device or Equipment To obtain the most recent readings or status updates from various devices or equipment, you need to query the point group that contains these values. The point group for a device or equipment can be retrieved using its filter. Once the point group is retrieved, you can request the latest values for all points within that group. See the `DS_GetPointGroup_Points` microflow from the *Building X Connector for Mendix* Sample App for details. This approach ensures that you have the most up-to-date information for monitoring and decision-making purposes. ### How to Ingest Sensor Readings and Sync Master Data This use case provides a guide on integrating external sensors, covering essential steps such as retrieving or creating equipment using the sensor ID, obtaining and handling point groups, managing points within these groups, and ingesting sensor measurements into corresponding points. These processes ensure integration and synchronization of sensor data, significantly enhancing the capability to effectively utilize sensor information. 1. Retrieve the equipment using the external ID, which corresponds to the sensor ID. 1. If the equipment does not exist, create it. 1. Obtain the point group using the equipment ID. This step can be repeated until the point group is created after a new device is created; point groups are created asynchronously, so there may be a slight delay. 1. Search for the point within the point group by its name. 1. If the point does not exist, create it within the point group for the equipment. 1. Ingest the sensor measurement value for the specified point, which belongs to the point group of the equipment representing the sensor. This process ensures that sensor readings are accurately captured and associated with the correct sensor metadata for further analysis and action. # Power BI Operations and Energy Dataset Explore the smart building and infrastructure operations dataset derived from the Building X Openness APIs. This page provides insight into various tables, their purpose, and detailed column descriptions for effective data analysis and management. ## Energy and Building Performance Dashboards in Power BI The Operations dataset serves as the foundation for the design of the Energy and Building Performance Dashboard within Power BI, leveraging the capabilities of both Power BI and the Building X platform. Its primary goal is to provide a holistic view of multiple facets associated with building portfolios, including information related to locations, equipment, devices, points, and point values. It leverages the following tables and functions: The following tables are exposed by the Operations API subscription: - **Devices**: This table contains information about various devices within the building, including their specifications and connections. - **Points**: This table contains individual data points collected from various building systems, such as sensors and meters. - **PointTags**: This table is designed to store information about tags associated with data points or measurements within the smart building and infrastructure system. These tags help categorize and provide context for data points. - **PointValues**: This table stores the actual values of data points collected over time. The following tables and functions are exposed by the Energy API subscription: - **EnergyMediumConsumptions**: This table contains information about energy medium consumption. - **EnergyMediumConsumptionValues**: This table stores the values of energy medium consumption over time. - **EnergyMediumConsumptionEmissions**: This table stores information about emissions associated with energy medium consumption. - **EnergyMediumConsumptionCosts**: This table stores information about costs associated with energy medium consumption. The following tables and functions are always exposed: - **Partitions**: This table is used to create and manage partitions, allowing for the organization and division of groups and subscriptions based on dataset access permissions. - **Locations**: This table stores information about different locations within a building, including their characteristics, contact details, and geographic coordinates. - **LocationTags**: This table is used to store information about tags associated with various physical locations within smart buildings and infrastructure. Tags provide additional details and categorization for locations. - **EquipmentTypes**: This table defines various types of equipment used within the building. It categorizes equipment types and provides descriptive details. - **Equipment**: This table contains detailed information about specific equipment within the building, such as devices, machinery, or systems. - **TimeTable**: This table displays time in various formats, including hour, minute, and second, as well as AM/PM format. It also includes labels for hour, minute, and second, a time key, and different hour and minute bins. A TimeTable is useful in Power BI for easy analysis and manipulation of time-based data, enabling detailed analysis and greater insights. Time intelligence functions in Power BI support efficient business analytics using periods or time frames. - **GenerateDateTable**: This function generates a comprehensive date table, incorporating various date-related columns such as year, month, day, quarter, week, and fiscal date attributes. Additionally, it calculates age, offsets, and includes user-specified options for the fiscal year start and the first day of the week. ### Key Metrics and Visualizations Create Power BI dashboards to track energy consumption trends and compare data across locations or equipment types. Evaluate building performance through KPI dashboards, considering factors like energy efficiency, temperature control, and occupancy. Assess equipment efficiency and pinpoint improvement opportunities. Implement device monitoring for prompt anomaly detection. Leverage historical data for in-depth analysis, trend identification, and informed decisions in energy management and building performance. - **Energy Consumption**: Create dashboards to monitor and analyze energy consumption trends, including historical data and comparisons between different locations or equipment types. - **Building Performance**: Develop KPI (Key Performance Indicator) dashboards to evaluate building performance based on factors like energy efficiency, temperature control, and occupancy. - **Equipment Efficiency**: Utilize equipment data to assess the efficiency of various equipment types and identify opportunities for improvement. - **Device Monitoring**: Implement real-time monitoring of devices and their data points to detect anomalies or issues promptly. - **Historical Analysis**: Leverage historical time-series data from devices to perform in-depth historical analysis, identify trends, and make informed decisions regarding energy management and building performance. ## Configuring Building X To access the Openness APIs, it is essential to create a dedicated machine user equipped with the appropriate roles and partitions. Only partitions assigned to the machine user can be used with the Building X Connector. You can manage machine users from the **Machine User** view within the **API Manager** application. Please refer to the **API Manager** User Guide for detailed instructions. ### Roles and Permissions The machine user should be associated with the following roles. Create user groups containing these roles and then assign these user groups to the machine user: - Structure API Machine User Read - Operations API Machine User Read ## Configuring Power BI Service This chapter describes the aspects of importing, configuring, and refreshing the Building X dataset within the Power BI service environment. ### Importing the Downloaded Dataset This chapter explains the process of importing the Building X Power BI dataset into the Power BI Web environment. You will learn how to configure parameters in the Settings view and how to update the dataset with the appropriate data source credentials. By following these steps, you can ensure that the dataset remains current and that you gain valuable insights from your data in Power BI. 1. Download the Power BI dataset (.pbix file) that you want to import into the Power BI service environment. 1. Open the Power BI service environment by navigating to the [Power BI service site](https://app.powerbi.com) and signing in with your credentials. 1. In the Power BI service environment, select the **My Workspace** tab on the left-hand side of the screen. 1. Select the **Import** button located at the top of the screen. 1. In the **Import a file** section, select the **Upload** button. 1. Select the downloaded Power BI dataset (.pbix file) from your computer and select the **Open** button. 1. After the file is uploaded, the dataset will be imported into the Power BI service environment. Wait for the import process to complete. 1. Once the import is complete, select the **Settings** option for the imported dataset. It can be found by hovering over the dataset and clicking on the ellipsis (...) button that appears. 1. In the **Settings** view, scroll down until you find the **Parameters** section. This section contains the parameters that need to be set up for the imported dataset. See the *Parameters* chapter for a description of the parameters. 1. After setting up all the parameters, select the **Apply** button to save the changes. ### Refreshing the Dataset This chapter explains how to refresh the Building X Power BI dataset in the Power BI Web environment. 1. To refresh the imported dataset, select the ellipsis (...) button that appears when hovering over the dataset in the Power BI service environment. 1. From the dropdown menu, select the **Refresh Now** option. This starts the refresh and data Building X retrieval process for the dataset. During the refresh process, Power BI may prompt you to provide data source credentials. In such cases, follow these steps: 1. For the data source credentials, select **Anonymous** as the authentication type. 1. Choose **Organizational** as the authentication type for the data source credentials. 1. Skip the testing of the connection by checking the provided checkbox. The behavior of **Anonymous** and **Organizational** data source credentials is as follows: - **Anonymous**: Selecting **Anonymous** as the data source credentials means that the dataset will connect to the data source without any user authentication. This is typically used when the data source allows anonymous access or when the dataset does not require any specific user credentials to retrieve the data. - **Organizational**: Choosing **Organizational** as the data source credentials means that the dataset will use the user's organizational account credentials to connect to the data source. This requires the user to have appropriate permissions and access rights to the data source within their organization. By selecting **Anonymous** and **Organizational** as the data source credentials, you can ensure that the dataset refreshes properly and accesses the required data from the data source based on the appropriate authentication type. ## Dataset Parameters This chapter describes the Power BI Operations dataset parameters. | **Parameter** | **Description** | **Data Type** | **Required** | | ---------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------- | ------------ | | **CustomerId** | This parameter represents the unique identifier of the customer or organization for which the data is being retrieved. It is used to ensure data access and authorization for the specific customer. The CustomerId can be found in the **Overview** view of the **Accounts** application. Obtain the CompanyId by accessing the Options menu of the **Company** tile. Select the **Copy Company Details** option and paste the contents of the clipboard into a text editor to obtain the Company Id. | Text | Yes | | **OperationsPartitionIds** | This parameter allows for data retrieval based on specific partition identifiers. Partitions represent logical divisions of data within the company. Multiple values can be specified, separated by semicolons, to retrieve data related to specific partitions, enabling more refined data selection. The PartitionId can be found in the **Machine User** view in the **API Manager** application by selecting the designated machine user and reviewing the details view. Only the first 10 partitions in the dataset will be used. This parameter is used to list partitions with the Operations API subscription. | Text | Yes | | **EnergyPartitionIds** | This parameter allows for data retrieval based on specific partition identifiers. Partitions represent logical divisions of data within the company. Multiple values can be specified, separated by semicolons, to retrieve data related to specific partitions, enabling more refined data selection. The PartitionId can be found in the **Machine User** view in the **API Manager** application by selecting the designated machine user and reviewing the details view. Only the first 10 partitions in the dataset will be used. This parameter is used to list partitions with the Energy API subscription. | Text | Yes | | **MachineUserClientId** | The Machine User Client ID is used for authentication when accessing the data. It ensures that the request is authorized and authenticated, allowing secure access to Building X Openness APIs. | Text | Yes | | **MachineUserSecret** | The Machine User key is a confidential authentication token used alongside the Machine User Client ID to verify and authorize the request. It adds an additional layer of security to Building X Openness. Machine users can be managed from the **Machine User** view in the **API Manager** app. | Any | Yes | | **PointValuesFrom** | This parameter specifies the start date for retrieving point values (time series data). Data is retrieved from this specified date forward, providing a way to filter and focus on specific time periods. See the *Limitations* chapter for point value limitations. | DateTime | Yes | | **PointTagsFilter** | This parameter filters the table of points and point values based on tag names and optional tag values. Specify multiple tags separated by semicolons. If both a tag name and a tag value are specified, separated by "=", only points with exact tag and value matches are included. Tag values are case-sensitive. If only a tag name is specified, any point containing that tag will be included. | Text | No | | **UseProductionEnvironment** | This parameter determines whether to use the production environment for data retrieval. When set to `true`, it indicates the use of the Building X production environment, ensuring data access from the production environment. If set to `false`, the dataset connects to the Building X test environment. | Logical (Boolean) | Yes | ## Limitations The following limitations apply. | **Limitation** | **Limit** | **Description** | | ------------------------------------------------- | --------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Maximum partitions** | 10 | To ensure responsive performance, the Power BI dataset imposes a limit that allows retrieval of a maximum number of partitions. | | **Maximum Data Point Values Stored** | 1,000,000 | To ensure responsive performance, the Power BI dataset imposes a limit that allows retrieval of a maximum number of data point values across all configured partitions. This limit is critical to prevent performance degradation, especially when working with large datasets. | | **Maximum Data Point Values per Point Retrieved** | 250,000 | To ensure responsive performance, the Power BI dataset enforces a limit on the number of data point values per individual data point. The most recent time series values are stored in the PointValue table. This limit is critical to prevent performance degradation, especially when working with large datasets. | | **Time Period** | 12 months | There is a limit to the maximum time frame for retrieving data point values. This limit applies regardless of the date specified in the PointValuesFrom parameter. | If parameters are changed, the data sources must be refreshed. See the *Refreshing the Dataset* chapter for more information. ## Openness API Subscription The Power BI dataset uses Openness APIs to retrieve data. Openness APIs are offered per call quota, so each time the Power BI dataset is refreshed, it consumes API calls and increases the quota counter. The number of API calls made depends on the setup of Building X, including partitions, devices, points, and data point values. **Consumed API calls** To accurately determine the number of API calls made by Power BI with each dataset refresh, follow these steps in Power BI Desktop. Please note that Power BI Service does not have a built-in feature for capturing diagnostics or detailed query performance statistics like Power Query in Power BI Desktop does. 1. Open the **Query Editor** in Power BI Desktop. 1. Go to the **Tools** page and select **Start Diagnostics**. 1. On the **Home** page, select **Refresh All**. 1. Go to the **Tools** page and select **Stop Diagnostics**. 1. In the **Queries** pane, navigate to the Diagnostics navigation tree and locate the table that starts with **Diagnostics_Detailed**. 1. Open this table and apply the following filters: 1. [Category] equals **Data Source**. 1. [Data Source Kind] equals **Web**. 1. [Resource] contains **eu.buildingx.siemens.com**. 1. On the **Transform** page, select **Statistics** and choose **Count Values**. The total number of API calls will now be displayed. ## Tags Tags in Building X are dynamic and powerful tools designed to streamline the categorization and filtering of data. They empower users with the flexibility to define custom tag names and assign values to facilitate the identification of locations, points, and devices within the building. The Building X Power BI dataset seamlessly converts all tags into separate columns, each with a `Tag_` prefix, improving data organization and accessibility. Additionally, when tag values are displayed as single-element lists, this feature intelligently converts them to single values. This simplifies data filtering and makes it more intuitive and user-friendly, ultimately enabling more accurate data analysis and efficient data management. In addition to the expanded Tag columns, the LocationTags and PointTags tables also enable filtering of the tag-owning tables based on the tag and its values. Tags without a value or with an empty text value are automatically set to `true` during conversion because these tags are considered marker tags. This simplifies filtering processes and ensures that tags without explicit values can be used for filtering. Tag names and values are case-sensitive. ## Troubleshooting This chapter describes problems and their solutions. ### Access to the resource is forbidden If you receive a "Forbidden Access" error when accessing the Power BI dataset, it indicates that one or more tables have failed to load. To resolve this issue, ensure that the following configuration items are correct in **API Manager** and **Power BI**: - Verify the **MachineUserClientId**, **MachineUserSecret**, **CompanyId**, and **PartitionIds** parameter values. - Verify the partitions assigned to the machine user in **API Manager**. See the *Configuring Building X* chapter for more information. ### The credentials provided for the Web source are invalid If you receive this error when accessing the Power BI dataset, it indicates that one or more tables have failed to load. To resolve this issue, ensure that the following configuration items are correct in **API Manager** and **Power BI**: - Check that the **MachineUserClientId** has the correct roles assigned. See the *Configuring Building X* chapter for more information. ### Data is missing When not all data is present in the tables, review the following documentation and configuration items: - Confirm expectations align with the information in the *Limitations* chapter. - Verify the accuracy of the **PartitionIds**, **PointValuesFrom**, and **PointTagsFilter** parameter values. ### Error: Unable to combine data The error message `[Unable to combine data] is accessing data sources that have privacy levels that cannot be used together. Please rebuild this data combination.` occurs when data sources with incompatible privacy levels are merged. To resolve this issue, review the data sources, adjust their privacy levels, and rebuild the data combination. This ensures data security and compliance. See the *Refreshing the Dataset* chapter for more information. ### Error: Tag column does not exist in the rowset The error message `The 'Tag_' column does not exist in the rowset` indicates that the specified Tag column cannot be found in the referenced table. This typically occurs when the tag is used as a filter in tables or dashboards but is no longer defined in the **Data Setup** platform application for the configured company or partitions. To resolve this, remove the referenced column from tables and dashboard filters. # Tables This chapter describes the Power BI Operations table and column definitions. ## Partitions The Partitions table is used to create and manage partitions, allowing for the organization and division of groups and subscriptions based on dataset access permissions. | **Column Name** | **Data Type** | **Description** | | --------------- | ------------- | -------------------------------- | | **Id** | text | Unique identifier for partitions | | **Name** | text | Name of the partition | | **Description** | text | Description of the partition | ## Locations The Locations table stores information about different locations within a building, including their characteristics, contact details, and geographic coordinates. | **Column Name** | **Data Type** | **Description** | | ----------------------- | ------------- | ----------------------------------------------------------------------------------------------------------------------------------- | | **Id** | text | Unique identifier for locations | | **PartitionId** | text | Identifier of the associated partition | | **Type** | text | Type of location (e.g., campus, building, room, floor) | | **Label** | text | Label or name for the location | | **TimeZone** | text | Time zone of the location | | **Description** | text | Description of the location | | **ExternalId** | text | External identifier for the location | | **Coordinate_X** | number | X-coordinate of the location | | **Coordinate_Y** | number | Y-coordinate of the location | | **PrimaryContactName** | text | Name of the primary contact for the location | | **PrimaryContactEmail** | text | Email of the primary contact | | **PrimaryContactPhone** | text | Phone number of the primary contact | | **HasPostalAddressId** | text | Identifier of associated postal address | | **Tag\_\*** | any | Each location-specific tag is represented as an individual column, making it effortless to access and filter data based on each tag | | **CreatedAt** | datetime | Timestamp of creation | | **UpdatedAt** | datetime | Timestamp of last update | | **CreatedBy** | text | Creator of the location | | **UpdatedBy** | text | Last updater of the location | Note: Prefiltered tables are also available for Campus, Building, Floor, and Room location types. ## LocationTags The LocationTags table is used to store information about tags associated with various physical locations within smart buildings and infrastructure. Tags provide additional details and categorization for locations. | **Column Name** | **Data Type** | **Description** | | --------------- | ------------- | ----------------------------------------------------------------------------------------------------------- | | **LocationId** | text | The identifier for a specific location | | **TagName** | text | The name of the tag associated with the location | | **TagValue** | any | The value of the tag, which can be of various data types, providing specific information about the location | ## EquipmentTypes The EquipmentTypes table defines various types of equipment used within the building. It categorizes equipment types and provides descriptive details. | **Column Name** | **Data Type** | **Description** | | ------------------- | ------------- | --------------------------------------------- | | **Id** | text | Unique identifier for equipment types | | **Type** | text | Type of equipment | | **PartitionId** | text | Identifier of the associated partition | | **Name** | text | Name of the equipment type | | **Description** | text | Description of the equipment type | | **Domain** | text | Domain or category of the equipment type | | **Deprecated** | text | Indicates if the equipment type is deprecated | | **Iri** | text | Model description for the equipment type | | **HasParentTypeId** | text | Identifier of the parent equipment type | | **HasParentType** | text | Type of the parent equipment type | | **CreatedAt** | datetime | Timestamp of creation | | **UpdatedAt** | datetime | Timestamp of last update | | **CreatedBy** | text | Creator of the equipment type | | **UpdatedBy** | text | Last updater of the equipment type | ## Equipment The Equipment table contains detailed information about specific equipment within the building, such as devices, machinery, or systems. | **Column Name** | **Data Type** | **Description** | | ---------------------- | ------------- | ----------------------------------------------- | | **Id** | text | Unique identifier for equipment | | **PartitionId** | text | Identifier of the associated partition | | **EquipmentTypeId** | text | Identifier of the associated equipment type | | **Name** | text | Name of the equipment | | **Notes** | text | Additional notes or information about equipment | | **ExternalId** | text | External identifier for equipment | | **SerialNumber** | text | Serial number of the equipment | | **Model** | text | Model information for the equipment | | **ManufacturingYear** | number | Year when the equipment was manufactured | | **ManufacturedBy** | text | Manufacturer of the equipment | | **InstallationDate** | datetime | Date when the equipment was installed | | **IsControlledById** | text | Identifier of the controlling equipment | | **IsControlledByType** | text | Type of the controlling equipment | | **IsPartOfId** | text | Identifier of the parent equipment | | **IsPartOfType** | text | Type of the parent equipment | | **FeedsId** | text | Identifier of equipment feeds | | **FeedsType** | text | Type of equipment feeds | | **LocationId** | text | Identifier of the associated location | | **LocationType** | text | Type of the associated location | | **CreatedAt** | datetime | Timestamp of creation | | **UpdatedAt** | datetime | Timestamp of last update | | **CreatedBy** | text | Creator of the equipment | | **UpdatedBy** | text | Last updater of the equipment | ## Devices The Devices table contains information about various devices within the building, including their specifications and connections. | **Column Name** | **Data Type** | **Description** | | ------------------- | ------------- | --------------------------------------------- | | **Id** | text | Unique identifier for devices | | **PartitionId** | text | Identifier of the associated partition | | **ModelName** | text | Name or model of the device | | **SerialNumber** | text | Serial number of the device | | **ProfileName** | text | Name of the device's profile | | **ProfileNotes** | text | Additional notes or details about the profile | | **HasGatewayType** | text | Type of the associated gateway | | **HasGatewayId** | text | Identifier of the associated gateway | | **HasLocationType** | text | Type of the associated location | | **HasLocationId** | text | Identifier of the associated location | ## Points The Points table contains individual data points collected from various building systems, such as sensors and meters. | **Column Name** | **Data Type** | **Description** | | ---------------------- | ------------- | -------------------------------------------------------------------------------------------------------------------------------------- | | **Id** | text | Unique identifier for data points | | **PartitionId** | text | Identifier of the associated partition | | **DeviceId** | text | Identifier of the associated device | | **EquipmentId** | text | Identifier of the associated equipment | | **PointGroupId** | text | Identifier of the associated point group | | **Name** | text | Name or label of the data point | | **Description** | text | Description of the data point | | **DataType** | text | Type of data collected (e.g., temperature, humidity) | | **IsActive** | logical | Indicates if the data point is active or not | | **Unit** | text | Measurement unit of the data point | | **SourceType** | text | Type of point source (e.g., PointNB) | | **Minimum** | number | Minimum value recorded for the data point | | **Maximum** | number | Maximum value recorded for the data point | | **Precision** | number | Precision or accuracy of the data point | | **CommandingSemantic** | text | Semantic information for commanding the point | | **Function** | text | Function or purpose of the data point | | **Tag\_\*** | any | Each point-specific tag is represented as an individual column, making it effortless to access and filter data based on each tag value | | **CreatedAt** | datetime | Timestamp of creation | | **UpdatedAt** | datetime | Timestamp of last update | ## PointTags The PointTags table is designed to store information about tags associated with data points or measurements within the smart building and infrastructure system. These tags help categorize and provide context for data points. | **Column Name** | **Data Type** | **Description** | | --------------- | ------------- | --------------------------------------------------------------------------------------------------------------- | | **PointId** | text | The identifier for a specific data point or measurement | | **TagName** | text | The name of the tag associated with the data point | | **TagValue** | any | The value of the tag, which can vary in data type, providing specific information or context for the data point | All tags defined with the data point are included, regardless of the value of the *PointTagsFilter* parameter, to reflect the data point configuration. ## PointValues The PointValues table stores the actual values of data points collected over time. | **Column Name** | **Data Type** | **Description** | | ------------------ | ------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Id** | text | Unique identifier for data point values | | **PartitionId** | text | Identifier of the associated partition | | **PointId** | text | Identifier of the associated data point | | **Timestamp** | datetime | Timestamp of the data point value | | **Value** | text | Actual value recorded at the timestamp | | **QualityOfValue** | numeric | Measurement reliability using the following values: 0: Good - Signifying high-quality, trustworthy data. 1: Bad - Generic Error - Denoting unclear or unspecified issues. 2: Bad - Communication Error - Indicating problems in data transmission. 3: Bad - Configuration Error - Relating to errors in system setup. 4: Bad - Off-Normal - Highlighting measurements outside expected parameters (user defined). 5: Bad - Gap - Identifying missing or incomplete data (user defined). | ## EnergyMediumConsumptions The EnergyMediumConsumptions table contains information about energy medium consumptions. | **Column Name** | **Data Type** | **Description** | | --------------- | ------------- | --------------------------------------------------------- | | **Id** | text | Unique identifier for energy medium consumptions | | **PartitionId** | text | Identifier of the associated partition | | **Name** | text | Name of the energy medium consumption | | **Disciplines** | record | Disciplines associated with the energy medium consumption | | **Reference** | record | Reference information for the energy medium consumption | ## EnergyMediumConsumptionValues The EnergyMediumConsumptionValues table stores the values of energy medium consumptions over time. | **Column Name** | **Data Type** | **Description** | | ----------------------- | ------------- | ------------------------------------------------------- | | **MediumConsumptionId** | text | Identifier of the associated energy medium consumption | | **PartitionId** | text | Identifier of the associated partition | | **Timestamp** | datetime | Timestamp of the energy medium consumption value | | **Value** | number | Actual value recorded at the timestamp | | **Unit** | text | Measurement unit of the energy medium consumption value | ## EnergyMediumConsumptionEmissions The EnergyMediumConsumptionEmissions table stores information about emissions associated with energy medium consumptions. | **Column Name** | **Data Type** | **Description** | | ----------------------- | ------------- | ---------------------------------------------------------- | | **MediumConsumptionId** | text | Identifier of the associated energy medium consumption | | **PartitionId** | text | Identifier of the associated partition | | **Timestamp** | datetime | Timestamp of the energy medium consumption emission | | **Value** | number | Actual value recorded at the timestamp | | **Unit** | text | Measurement unit of the energy medium consumption emission | ## EnergyMediumConsumptionCosts The EnergyMediumConsumptionCosts table stores information about costs associated with energy medium consumptions. | **Column Name** | **Data Type** | **Description** | | ----------------------- | ------------- | ------------------------------------------------------ | | **MediumConsumptionId** | text | Identifier of the associated energy medium consumption | | **PartitionId** | text | Identifier of the associated partition | | **Timestamp** | datetime | Timestamp of the energy medium consumption cost | | **Value** | number | Actual value recorded at the timestamp | | **Unit** | text | Measurement unit of the energy medium consumption cost | # Security Manager Access PACS SDK Building X's existing customers rely on on-premises legacy Physical Access Control Systems (PACS). These systems hold valuable data, which we can import. For more details, please have a look at the SDK documentation. The Security Manager Access PACS SDK is a comprehensive Software Development Kit tailored for software developers from both partners and end customers. It caters to those seeking to seamlessly integrate third-party access control systems into Building X. ## Key Benefits Our PACS SDK serves as a robust toolset, enabling smooth and efficient integration of third-party access control systems into Building X. - The PACS SDK allows the integration of any third-party access control system into Building X, tailored to meet diverse customer requirements. - Experience advanced access capabilities to enrich your software applications. - One SDK provides a unified approach for seamless integration with third-party access control systems. - Detailed documentation supports developers throughout the development process. ## Request To request the Security Manager Access PACS SDK, please use the [form](https://forms.office.com/r/eenQfQQvy3). # Security Manager Access Mobile SDK The Security Manager Access Mobile SDK from Siemens empowers you to integrate access functionality into your own mobile applications. With this SDK, your custom mobile apps can unlock doors by simply holding the mobile device up to a reader. The SDK is available for both Android and iOS platforms. ## Key Benefits Make the most of the advanced access functionality offered by our Mobile SDK to enhance your mobile applications. Seamlessly integrate Siemens security solutions for a superior user experience. - One SDK for seamless integration with all vendor systems. - Easy-to-use API for quick and efficient integration. - Comprehensive documentation to support development. ## Request To request the Security Manager Access Mobile SDK, please use the [form](https://forms.office.com/r/aAAGCc9DYG). # Industrial Operations X documentation # Contact ## Get to work with Industrial Operations X Ready to make your production more adaptable and autonomous? Get in touch with us now. [Contact Us](https://www.siemens.com/global/en/general/contact.html) ## Discover the Digital Enterprise Make your production a Digital Enterprise and combine the real and digital worlds. [More about digital enterprise](https://www.siemens.com/global/en/company/topic-areas/digital-enterprise.html) - [Overview](overview.html) - [Industrial Operations X Products](products.html) - [Industrial Operations X APIs in API Catalog](/apis.html?tag=Industrial+Operations+X) - [Contact](contact.html) # Industrial Operations X With smart IT capabilities and software development methods, Industrial Operations X helps make your production process more flexible, autonomous and people-centric. This enables teams to overcome growing challenges in the industry sector. [Discover Products](products.html) [Explore APIs](/apis.html?tag=Industrial+Operations+X) ## What is Industrial Operations X Industrial Operations X enables a highly flexible and people-centric production. We integrate IT and software capabilities into automation, with machine learning and data analytics built-in to increase adaptability. Our intuitive automation solutions support teams from a range of automation and IT backgrounds. ## Industrial Operations X - a true Siemens Xcelerator Solution Industrial Operations X is part of Siemens Xcelerator, focusing on industrial operations. [Siemens Xcelerator](../resources/how-tos/overview.html) is an open digital business platform that enables digital transformation faster and at a larger scale. It is based on four principles: - **Interoperable**: Several solutions work together across communication structures with more to be integrated in the future - **Flexible**: low-code environment for easy customization of solutions - **Open**: standardized APIs for powerful data analysis - **As a service**: Several options are already available with more to be added as the platform grows ## An open and interoperable ecosystem Industrial Operations X helps build the open ecosystem of Siemens Xcelerator, as Industrial Operations X products can be combined with existing Siemens automation solutions. As an open and interoperable system, it can interact with 3rd party systems in industrial operations, and other providers are able to join the ecosystem. Our commitment to the open ecosystem is reflected in the [product overview with all products and APIs](products.html) which are part of Siemens Industrial Operations X. (we are constantly onboarding the documentation of new products and APIs, so please check back often!) You can also find the list of all [Industrial Operations X APIs](/apis.html?tag=Industrial+Operations+X) in our API catalog. ## Flexibility that responds to your needs Siemens Xcelerator promises to make digitalization easy, open and flexible. Industrial Operations X makes this promise real in the production and automation world. Integrating IT and software capabilities into the shop floor enables you to create a highly flexible and people-centric production capable of responding to new requirements. ## A holistic approach for a bigger picture Industrial Operations X focuses on consistently and holistically integrating IT capabilities and proven methods of software operations into the world of industrial automation — from production to the processes of suppliers, customers and data from buildings and grids. Maintaining an overview of all areas of industrial operations gives you a more accurate picture of what’s happening in the production chain, and you can respond accordingly. # Industrial Operations X Products Industrial Operations X helps build the open ecosystem of Siemens Xcelerator, as Industrial Operations X products can be combined with existing Siemens automation solutions. As an open and interoperable system, it can interact with 3rd party systems in industrial operations, and other providers are able to join the ecosystem. Our commitment to the open ecosystem is reflected in the list below, with all products and APIs which are part of Siemens Industrial Operations X. (we are constantly onboarding the documentation of new products and APIs, so please check back often!) You can also find the list of all [Industrial Operations X APIs](/apis.html?tag=Industrial+Operations+X) in our API catalog. # ANYbotics documentation # Contact For any questions related to the ANYbotics, please contact [info@anybotics.com](mailto:info@anybotics.com). - [Overview](overview.html) - [Contact](contact.html) # ANYmal API and SDK ## Introduction ANYbotics offers robotic solutions for automated industrial inspections. These robotic solutions perform routine condition monitoring tasks based on our legged robotic platforms, ANYmal and the Ex-certified ANYmal X. ANYmal enables operators to automate on-site inspections, increasing the safety of plant workers and improving data quality. In addition to the ANYbotics "Workforce" user interface for robot control and mission management, ANYbotics provides an API (Application Programming Interface) for integration with third-party software. With the ANYmal API, you can automatically transfer inspection data, trigger robot missions, and manage your robot fleet directly from your existing asset management or digital twin software. The ANYmal API allows developers to harness the power of ANYmal to seamlessly integrate data into their infrastructure. ## API Capabilities The ANYmal API integrates your automated inspection coworker, ANYmal, into your mission and data management infrastructure. The API functionality includes: - **Real-Time Data Streaming**: Acoustic, thermal, gas, and visual inspection feedback. - **Mission Management**: Trigger, monitor (pause/resume/stop), and modify pre-scheduled and “click & act” missions. - **Detailed Feedback**: ANYmal’s pose, joint positions, and location mapping, as well as mission-critical data for post-mission analyses. - **Mission Notifications**: Timely updates on mission status, including completion alerts and anomaly reporting. - **Connectivity Management**: Stable performance during intermittent connectivity issues through connection status monitoring and on-robot caching. - **Battery Level Monitoring**: Track the power status to ensure uninterrupted autonomous operations. - **Safety & Security**: Incorporate third-party authentication via OAuth, emergency and protective stop functions, and authority management for secure access control. - **Telepresence**: Support real-time visual feedback through WebRTC-based streaming video and provide a direct command interface for manual navigation or task-specific commands. These features ensure a robust platform for users to effectively deploy ANYmal for various inspection tasks and robot operations, providing access to status, health information, and live video. ## ANYmal Server Integration ANYmal communicates with an ANYmal server, which offers various application possibilities. The ANYmal server allows developers to route data through the cloud infrastructure provided by ANYbotics, their own cloud networks, or local networks as required. As a customer or partner, you can integrate ANYmal with your own and third-party software systems for asset management, the digital twin, or the data portal, enabling bi-directional communication. ## Development ANYbotics sets up the data exchange via the ANYmal API with actionability and ease of use in mind: You get time-stamped and position-referenced inspection insights and receive the information in standard file formats such as JPG, WAV, E57, PLY, and DAE. The ANYmal API is based on an RPC (Remote Procedure Call) API using gRPC. It is a high-performance RPC framework, using HTTP/2 for transport and Protocol Buffers for interface description. ANYbotics also provides an SDK (Software Development Kit) to simplify interaction with the ANYmal API in C++ and Python. You can access the code samples and comprehensive documentation for the ANYmal SDK. With SDK or gRPC documentation access, developers can choose how to interface with ANYmal API to control missions, receive data, and monitor ANYmal. ## Developer Resources ANYmal SDK code samples and complete documentation - [support team](https://support.anybotics.com/request). The [ANYmal Research Community](https://www.anymal-research.org/) develops and shares software and other information. # Battery Passport documentation # Changelog The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). ## [0.0.1] - 2024-02-06 ### Added - Initial version of 'Battery Passport REST API' # Contact For any technical questions related to the Battery Passport REST API, please contact **API Mailbox**\ Battery Passport REST API Mailbox\ Email: [battery-passport.industry@siemens.com](mailto:battery-passport.industry@siemens.com) # FAQ ***What is the difference between the public and private passport's attributes?*** The public path `passport/public/{batterypassid}` accepts any requests (no authorization needed), because of the European commission regulation requirement, which needs to provide at any time public available passport data. # Getting Started Getting started with using **Battery Passport API** involves the following steps: 1. Request Battery Passport API access from the Battery Passport team via [battery-passport.industry@siemens.com](mailto:battery-passport.industry@siemens.com) 1. After our approval process you will get Client ID and a Client Secret. 1. Make a request against our identity provider SiemensID to receive a JWT by using Client ID and Client Secret provided by Battery Passport team. 1. Include the JWT Token `Authorization` header of your HTTP requests. ## Request a token ```bash export CLIENT_ID= export CLIENT_SECRET= curl https:///oauth/token \ <-- You will get the correct url from us -H 'content-type: application/json' \ -d "{ \"client_id\":\"$CLIENT_ID\", \"client_secret\":\"$CLIENT_SECRET\", \"audience\":\"https://api.battery-passport.siemens.com\", \"grant_type\":\"client_credentials\" }" ``` To run this example yourself, set the `CLIENT_ID` and `CLIENT_SECRET` first. ### Example response ```json { "access_token": "{jwtHeader}.{jwtPayload}.{jwtSignature}", "token_type": "Bearer", "expires_in": 86400 } ``` The token, or JWT (JSON Web Token), is the value of the `access_token`-property in the response. You can now use it by passing it in the `Authorization`-header of any subsequent API requests. The `expires_in`-property represents the number of seconds your token is valid, usually, the value corresponds to 24 hours. When this time has elapsed you will need to create a new token. Make sure that at least the scope `application.bp_portal.data_viewer` is in the list of scopes. ## Example request to Battery Passport APIs ```bash curl --request GET \ --url https://api.battery-passport.siemens.com/v1/passports/public/{batterypassid} \ --header 'Authorization: Bearer {jwtHeader}.{jwtPayload}.{jwtSignature}' ``` The response contains the current weather. The resulting JSON looks like ```json { "status": { "state": "active" }, "general_information": { "identification": { "uid": "string", "manufacturer_id": { "name": "string", "contact": { "fax_number": "string", "website": "string", "phone_number": "string", "email": "string" }, "address": { "locality": "string", "country_sn": "string", "post_code": 0, "street": "string", "number": 0, "premise": "string", "postal_delivery_point": "string" } }, "manufacturing_date": "string", "manufacturing_place": { "locality": "string", "country_sn": "string", "post_code": 0, "street": "string", "number": 0, "premise": "string", "postal_delivery_point": "string" } }, "characteristics": { "category": [ "string" ], "weight": 0 } }, "compliance": { "conformity": { "declaration_of_conformity_eu": { "name": "string", "description": "string", "location": "string" }, "id_declaration_of_conformity_eu": "string" }, "symbols": { "collection_symbol": { "name": "string", "description": "string", "location": "string" }, "cadmium_symbol": { "name": "string", "description": "string", "location": "string" }, "lead_symbol": { "name": "string", "description": "string", "location": "string" } } }, "materials": { "components": [ { "critical_raw_materials": [ "string" ], "hazardous_substances": [ { "substance": { "name": "string", "class": "string", "category": "string", "id": "string", "substance_impact": "string" }, "concentration_range": 0, "location": "string" } ] } ], "materials": { "chemistry": "string" } }, "carbon_footprint": { "carbon_footprint": 0, "pre_processing_share_of_carbon_footprint": 0, "production_share_of_carbon_footprint": 0, "distribution_share_of_carbon_footprint": 0, "end_of_life_share_of_carbon_footprint": 0, "carbon_footprint_performance_class": "string", "web_link_to_carbon_footprint_study": { "name": "string", "description": "string", "location": "string" } }, "supply_chain_due_diligence": { "due_diligence_report": { "information_of_due_diligence_report": { "name": "string", "description": "string", "location": "string" } }, "additional_voluntary": { "third_party_supply_chain_assurances": { "name": "string", "description": "string", "location": "string" }, "eu_tax_disclosure_statement": { "name": "string", "description": "string", "location": "string" }, "sustainability_report": { "name": "string", "description": "string", "location": "string" } } }, "circularity": { "safety_requirements": { "extinguishing_agent": [ "string" ] }, "recycled_content": { "pre_consumer_recycled_nickel_share": 0, "pre_consumer_recycled_cobalt_share": 0, "pre_consumer_recycled_lithium_share": 0, "pre_consumer_recycled_lead_share": 0, "post_consumer_recycled_nickel_share": 0, "post_consumer_recycled_cobalt_share": 0, "post_consumer_recycled_lithium_share": 0, "post_consumer_recycled_lead_share": 0 }, "renewable_content": { "renewable_content_share": 0 }, "end_of_life_information": { "role_of_end_users_to_waste_prevention": { "name": "string", "description": "string", "location": "string" }, "role_of_end_users_to_separate_collection": { "name": "string", "description": "string", "location": "string" }, "information_on_recycling_operations": { "name": "string", "description": "string", "location": "string" } } }, "performance": { "capacity": { "certified_usable_energy": 0, "rated_capacity": 0, "nominal_voltage": 0, "min_voltage": 0, "max_voltage": 0 }, "power_capability": { "original_power_capability": 0, "max_permitted_power": 0, "ratio_between_power_and_energy": 0 }, "round_trip_energy": { "initial_round_trip_energy_efficiency": 0, "round_trip_energy_efficiency_at_half_life": 0 }, "internal_resistance": { "initial_resistance": { "pack_id": "string", "resistance": 0, "modules": [ { "module_id": "string", "resistance": 0, "cells": [ { "cell_id": "string", "resistance": 0 } ] } ] } }, "battery_lifetime": { "expected_lifetime_n_charge_discharge_cycles": 0, "cycle_life_reference_test": "string", "c_rate_of_cycle_life_test": 0, "capacity_threshold_for_exhaustion": 0, "soce_threshold_for_exhaustion": 0, "warranty_period": 0 }, "temperature_conditions": { "lower_temperature_idle_state": 0, "upper_temperature_idle_state": 0 } } } ``` - [Overview](overview.html) - [Getting Started](getting-started.html) - [Battery Passport Rest API](rest-api-spec.html) - [Changelog](changelog.html) - [FAQ](faq.html) - [Contact](contact.html) # Overview ## Battery Passport API The Battery Passport platform offers a rest API in order to provide data exchange in a standardized and self-described way. It is essential to provide a machine to machine interface in order to seamlessly integrate any automated process. ## Battery Passport API functionalities The Battery Passport platform provides operations for two different subsystems: - User management - Passport management ### User management subsystem functionalities The main functionalities for a user are: - Get user information (from the logged user) - Update user information (in case the logged user has the proper permissions or rights) - Create user (in case the logged user has the proper permissions or rights) ### Passport management subsystem functionalities The main functionalities to manage passports are: - Get public passport information (for any logged or not logged user requiring the information) - Get owned passports (for any logged user with the proper permissions or rights) - Get owned passport information (for any logged user with the proper permissions or rights) - Create passport (for any logged user with the proper permissions or rights) - Update owned passport (for any logged user with the proper permissions or rights) - Delete owned passport (for any logged user with the proper permissions or rights) ## Contact For general information about the application, the APIs, any issues or feature requests please contact: Email: [battery-passport.industry@siemens.com](mailto:battery-passport.industry@siemens.com) # Battery Passport REST API - [Download Postman Collection](postman-collection.json) # COMOS Web REST API documentation # Contact For any technical questions related to the COMOS Web REST API, please contact **DI PA SW Lifecycle 1**\ Development team COMOS Web REST API\ Email: [lifecycle-1.industry@siemens.com](mailto:lifecycle-1.industry@siemens.com) # FAQ ***Where do I get the sessionId that is needed for calling the endpoints?*** Before calling any other endpoint, you need to log in through sessions service (`public/api/sessions/v1/sessions/actions/login`). The result will contain the sessionId you need for accessing any further resources. Please see the [Get Started](get-started.html) page for further details. # Get Started Once the web server is running, the API can be called. If you want to experiment with the API using Swagger, you can visit ":/swagger/index#/". The recommended procedure for using the API is as follows. 1. The client calls the discovery endpoint. The discovery endpoint gives information on the available features and should be used to discover the actual URLs of the other features. Please refer to the section on [Discovery](get-started.html#discovery) for further details. 1. The client logs in. On a successful login, a session ID is returned. This session ID has to be provided for almost all the remaining calls to the COMOS Web REST API. The client can provide the session ID to the COMOS Web REST API in one of the following ways: - Via the HTTP header `COMOS-API-Session` - Via an HTTP cookie-header with a cookie named `Comos.Api.Session` Note that this session ID is not an authentication token. Nevertheless, it can only be used by the same account that was used to log in. Please refer to the section on [Authentication and Authorization](get-started.html#authentication-and-authorization) for further details. Example of a response from the "public/api/sessions/v1/sessions/actions/login" endpoint ```json { "SessionId": "3797870ad22g44e6ra83fd72e84d2a09", "UserName": "your user name", "HeartbeatInterval": 60 } ``` 1. The client calls the features of the COMOS Web REST API. The client must use the relative paths provided by the discovery to assemble the target URL, and it usually has to provide the session ID in one of the above-mentioned ways. 1. The client logs out. ## Discovery The available features of the COMOS Web REST API can be called and addressed via its "discovery" feature. Each feature of the COMOS Web REST API has its own version. This enables a client to decide about the [compatibility](versioning.html) for each feature individually and hence give detailed feedback to the user. A breaking change in a feature, leading to incompatibility with previous versions, does not influence the whole API but just this specific feature. The available features can be requested via an "HTTP GET" request at the following path: "public/api/discovery/v1/services" Example of a response from the "public/api/discovery/v1/services" endpoint ```json [ { "Name": "projects", "Versions": [{"VersionNumber": "1.0.0", "Path": "api/projects/v1"}] }, { "Name": "discovery", "Versions": [{"VersionNumber": "1.0.0", "Path": "api/discovery/v1"}] }, { "Name": "users", "Versions": [{"VersionNumber": "1.0.0", "Path": "api/users/v1"}] }, { "Name": "sessions", "Versions": [{"VersionNumber": "1.0.0", "Path": "api/sessions/v1"}] } ] ``` The response may vary Note that the actual content of the result will vary according to the release of the COMOS Web REST API. The `Name` property contains the name of the feature. This name will not change during the evolution of the API. The `Versions` property contains a list of all currently available versions for this feature. A feature might be available in multiple versions. Each version has a `VersionNumber` in the Semantic versioning format as well as the path to the feature. This is the relative path, which has to be added to the base address of the COMOS Web REST API, including the "public". Keywords used The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119. A client SHOULD use `discovery` to get the actual URL of a specific service. The only feature whose relative URL is fixed is the `discovery` feature with the MAJOR version 1. Hence, a client SHOULD call `discovery` before sending another API request. A client SHOULD NOT use a fixed reference to a specific URL of the API except the URL used for `discovery`. ## Authentication and Authorization - The Rest API supports "Windows authentication" and "Two-factor authentication" via OpenId Connect. - Swagger only supports Windows authentication. - All requests to the Rest API must include valid headers, e.g.: - Comos-Api-Session: The session id received when using the login endpoint - A cookie header containing a cookie called `Comos.Api.Session` # Logging ## Protocol function Protocol or log functions work similarly to COMOS, but you must use `bin\Comos.Web.Worker.exe.config` and not `bin\Comos.exe.config`. As this file is located on the server, logging settings can only be changed there. ## Protocol function for the web page In both configuration files "\\Web\\web.config" and "\\ComosServices\\ComosGateway\\Hosting\\Web.config" the following lines have to be set to active: ```xml --> ``` And under \: ```xml ``` Warning Verbose logging should be used with care. - [Overview](overview.html) - [Get Started](get-started.html) - [COMOS Web REST API](rest-api-spec.html) - [FAQ](faq.html) - [Logging](logging.html) - [Versioning and Compatibility](versioning.html) - [Contact](contact.html) # Overview COMOS for the integrated management of plant projects ensures that engineers and operators can access all project-relevant data at all times, across all company levels, and in all project phases. It offers a seamless flow of information by providing a common database. COMOS lays the foundation for greater reliability in decision-making and more efficient processes throughout the entire plant. [Additional Information on COMOS](https://xcelerator.siemens.com/global/en/all-offerings/products/c/comos.html) ## Introduction The COMOS Web REST API is available for COMOS Web. It is a REST-style, HTTP-based API used to interact with COMOS data and to access COMOS features. This will give you the option, for example, to reuse parts of COMOS Web in your own web client or to use it for non-interactive data exchange. The API supports standard operations like Create, Read, Update, Delete (CRUD) and is used for dedicated COMOS functionalities, e.g.: - Creating working layers and COMOS objects - Reading project / working layer information - Reading, updating and deleting COMOS objects - Reading document revision information - Downloading / uploading documents - Performing check-ins, check-outs and revision steps on documents [Additional Information on COMOS Web](https://xcelerator.siemens.com/global/en/all-offerings/apis/c/comos-web-api.html) # COMOS Web REST API - [Download OpenAPI Specification](comos-rest-apispec.json) # Versioning and Compatibility The COMOS Web REST API will evolve over time. Additional features will be added and existing features might be extended, adjusted or even removed. Likewise, the API clients will evolve. A client using the COMOS Web REST API might not be deployed or updated at the same time as the API. Instead, a client might be updated before the COMOS Web REST API is updated and vice versa. Different versions of a client might be used at the same time. An older client might not know about new COMOS Web REST API features but should be able to use older features or even to detect that a feature might not be available anymore. A newer client should be able to decide which features are already supported by the COMOS Web REST API. The API uses a versioning strategy to enable the following: - Both the COMOS Web REST API and the clients using the API shall be able to evolve independently to a certain extent. - Clients shall be able to detect existing API features during run-time and, optionally, to provide feedback on this.\ (For example, by deactivating control elements in a user interface) ## Semantic Versioning Semantic Versioning is used to version features of the COMOS Web REST API. A version number is indicated in the following format: MAJOR.MINOR.PATCH Each part is a non-negative integer consisting of digits only and without leading + or leading zeros. Example: 1.0.5 Given a version number MAJOR.MINOR.PATCH, increment the: 1. MAJOR version when you make incompatible API changes 1. MINOR version when you add functionality in a backwards-compatible manner 1. PATCH version when you make backwards-compatible bug fixes Unsupported version numbers Version numbers deviating from the following are not supported. - Version numbers for preliminary publications.\ Example: 1.0.0-alpha - Version numbers with build metadata.\ Example: 1.0.0+20130313144700 Keywords used The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119. ## Versioning and client behavior A client SHOULD implement the following behavior. If a client has been built against a specific feature version, e.g. version 2.3.5, the following compatibility is assured: ### The MAJOR version on the server is greater - The client MUST NOT use the server version. It MAY notify the user about this fact, e.g., by displaying a message or by deactivating or removing specific user controls. ### The MAJOR version on the server is equal - The MINOR version on the server is greater.\ The client can use the feature and all the endpoints known to the client. The functionality of the feature has been extended since the client was implemented, but only in a compatible way. - The MINOR version on the server is equal.\ The client can use the feature and all the endpoints known to the client. The functionality of the feature has not changed. - The MINOR version on the server is smaller.\ The client can use the feature, but only the subset that has already been available up to the server version. ### The MAJOR version on the server is smaller - The client can use the feature, but only the subset that has already been available up to the server version (including the MINOR version). A change of the PATCH version does not affect the compatibility. If a new MAJOR version of a feature is available, a new client implementation SHOULD adjust to the new version. For compatibility reasons, a feature MAY be available in multiple versions temporarily, usually in the current MAJOR version and the previous MAJOR version, e.g., 1.7.3 and 2.0.1. If a feature is available in multiple versions and a client discovering the features can handle multiple versions—especially MAJOR versions—, the client SHOULD use the newer version as this is generally an improvement of the previous versions. Example: Compatibility scenario for a client that has been built against version 2.3.5 - The feature is available in versions 3.1.4, 2.3.5, 2.7.0 and 1.8.2. The client SHOULD use version 2.7.0. (It is compatible with 2.3.5 but newer.) It MUST NOT use version 3.1.4. (Version 3.1.4 comprises a breaking change.) - The feature is available in versions 2.4.7 and 2.4.8. The client SHOULD use version 2.4.8. (Both 2.4.7 and 2.4.8 are compatible with 2.3.5, but 2.3.8 is newer.) - The feature is available in version 3.1.4. The client MUST NOT use the available service. - The feature is available in version 1.4.5. The client MAY use the available service. But from a client perspective, it MAY also be valid to decide that this server version is not supported anymore. Note Typically, a feature will just be available in multiple MAJOR versions. Still, clients MUST be able to handle various MAJOR, MINOR or PATCH versions. Clients MUST ignore versions with invalid version numbers Example of invalid version numbers: - 1.0, 1.0.0-alpha - 01.0.0, a.0.0 - 1e2.0.0, -1.0.0, 1 A client MUST be able to handle unknown additional properties — e.g., in a JSON response — without breaking if the MINOR version of the feature has increased since implementation of the client. # DepotFinity documentation # Authorization Authorization provides you the access to consume DepotFinity APIs. To set up authorization and consume DepotFinity APIs, perform the following steps. 1. Request credentials for machine access from Siemens Regional Support. The support team provides the client credentials such as `clientId` and `clientSecret`. 1. Create a token using the client credentials to start using DepotFinity APIs. See [Token Management](TokenManagementAPI/token_management_api.html), to learn how to create a token. 1. Pass the `TOKEN` in the authorization header to access DepotFinity APIs. Warning DO NOT share or store your token in source control. Make sure if the token is stored safely. Anyone with access to your token can access the system on your behalf. Note Credentials are valid only for the requested APIs. To access additional APIs, contact **Siemens Regional Support**. # Contact [Visit DepotFinity](https://www.siemens.com/en-us/products/emobility/depotfinity/) or [Contact our experts](https://www.siemens.com/en-us/products/emobility/) to maximize the uptime of your electrified fleet with our use-case optimized DepotFinity Developer APIs. # General Information ## DepotFinity APIs DepotFinity API documentation aims to describe the specifications, conventions, steps to get started, and how to use DepotFinity APIs. The DepotFinity APIs are based on the [JSON:API](https://jsonapi.org/) specification with some customizations. You must be authorized to use the DepotFinity APIs. For more information on authorization, see [Authorization](authorization.html). Access to API's will be available only when subscribed to the relevant packages, see [DepotFinity Packages](#depotfinity-packages) The sample requests and responses within this document use the following configuration. - Linux/MacOS shell in which environmental variables are set using the `export-command`. Other environments may require a different setup. For example, Windows uses the `set-command`. - Example API calls are demonstrated using "curl" command line utility. However, the APIs can be used in any programming language with an HTTP client such as Go, Python, NodeJS, Javascript, and Java. DepotFinity APIs allow you to perform the following functions. - Manage tokens - Manage transactions - Control charging power limits - Generate session reports - Get energy prices and power consumption - Manage RFIDs - Send remote commands to the charger - Manage Reservations - Terminate an active charger connection - Interface with EV using VDV-261 parameters (The Authorization procedure for VDV-261 API is different from other DepotFinity APIs. For more information see [VDV-261](VDV261/vdv261_api.html)) ## DepotFinity Packages Note Access to API's will be available only when subscribed to the relevant packages. | API | Basic | Advanced | Premium | | --------------------------------------------------------------------------------------------------- | ----- | -------- | ------- | | [Charger Status API](ChargerStatusAPI/charger_status_api.html) | ✅ | ✅ | ✅ | | [Transactions API](TransactionManagementAPI/transaction_management_api.html) | ✅ | ✅ | ✅ | | [Session Reports API](SessionReportsAPI/session_reports_api.html) (applicable to only HPC chargers) | ✅ | ✅ | ✅ | | [Time Series API](TimeSeriesAPI/time_series_api.html) | ✅ | ✅ | ✅ | | [Force Termination API](ForceTerminateConnectionAPI/force_terminate_connection_api.html) | ✅ | ✅ | ✅ | | [RFID Management API](RFIDManagementAPI/rfid_management_api.html) | | ✅ | ✅ | | [Remote Commands API](RemoteCommandAPI/remote_command_api.html) | | ✅ | ✅ | | [Reservations API](ReservationsAPI/reservations_api.html) | | ✅ | ✅ | | [Charger Control API](ChargerControlAPI/charger_control_api.html) | | | ✅ | | [Virtual Power Plant API](VPPAPI/vpp_api.html) | | | ✅ | | [VDV-261 API](VDV261/vdv261_api.html) | | | ✅ | ## HTTP Status Codes HTTP response status codes indicate if a specific HTTP request is successful or not. | Term | Description | | ---- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 200 | Standard response for successful HTTP requests. The actual response will depend on the request method used. In a GET request, the response will contain an entity corresponding to the requested resource. In a POST request, the response will contain an entity describing or containing the result of the action. | | 201 | The request has been fulfilled, resulting in the creation of a new resource. | | 202 | The request has been accepted for processing, but the processing has not been completed. The request might or might not be eventually acted upon, and may be disallowed when processing occurs. | | 204 | The server successfully processed the request and is not returning any content. | | 400 | The server cannot or will not process the request due to an apparent client error. For example, malformed request syntax, size too large, invalid request message framing, or deceptive request routing. | | 401 | Similar to 403 Forbidden, but specifically for use when authentication is required and has failed or has not yet been provided. | | 403 | The request was valid, but the server is refusing action. The user might not have the necessary permissions for a resource, or may need an account of some sort. This code is also typically used if the request provided authentication via the WWW-Authenticate header field, but the server did not accept that authentication. | | 404 | The requested resource could not be found but may be available in the future. Subsequent requests by the client are permissible. | | 405 | A request method is not supported for the requested resource. For example, a GET request on a form that requires data to be presented via POST, or a PUT request on a read-only resource. | | 406 | The requested resource is capable of generating only content not acceptable according to the Accept headers sent in the request. | | 409 | Indicates that the request could not be processed because of conflict in the current state of the resource, such as an edit conflict between multiple simultaneous updates. | | 429 | The user has sent too many requests in a given amount of time. Intended for use with rate-limiting schemes. *Note: Allows 100 queries per five minutes per IP address* | | 500 | A generic error message, given when an unexpected condition was encountered and no more specific message is suitable. | | 503 | The server cannot handle the request (because it is overloaded or down for maintenance). Generally, this is a temporary state. *Note: Approximately 90 seconds* | | 504 | The server was acting as a gateway or proxy and did not receive a timely response from the upstream server. | ## Concepts & Glossary The following table lists the terms, concepts, and glossary of different parameters used in DepotFinity APIs. | Term | Description | | ---------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Charger ID | A unique identifier that represents a charger in the network of chargers. | | Connector ID | A unique identifier that represents a connector in the charger. There may be more than one Connector ID's linked to a charger. | | Driver ID | A unique identifier that represents a driver linked to an EV participating in the charging activity. | | Energy Delivered | Total energy delivered by the charger to increase the vehicle SoC. | | EV | Electric Vehicle | | JWT | JSON Web Token | | MAC ID | A media access control address is a hexadecimal identifier assigned to each device connected to the network. | | RFID | Radio frequency identifier is used in devices called RFID tags. The RFID tag can be used for the purposes of identification, authorizing charging activity, and so on. | | SoC | State of charge of electric vehicle's battery. | | Transaction ID | A unique identifier that represents a specific charging event at a specific time. | | Vehicle ID | A unique identifier that represents the electric vehicle participating in the charging activity. | | VIN | Vehicle Identification Number | | HPC | High Power Charging | - [Overview](overview.html) - APIs & Services - [General Information](general_information.html) - Authorization - [Overview](authorization.html) - [Quick Start](TokenManagementAPI/token_management_api.html) - [API Reference](TokenManagementAPI/token_management_api_reference.html) - Charger Status API - [Overview](ChargerStatusAPI/charger_status_api.html) - [API Reference](ChargerStatusAPI/charger_status_api_reference.html) - Transactions API - [Overview](TransactionManagementAPI/transaction_management_api.html) - [API Reference](TransactionManagementAPI/transaction_management_api_reference.html) - Session Reports API - [Overview](SessionReportsAPI/session_reports_api.html) - [API Reference](SessionReportsAPI/session_reports_api_reference.html) - Time Series API - [Overview](TimeSeriesAPI/time_series_api.html) - [API Reference](TimeSeriesAPI/time_series_api_reference.html) - Force Termination API - [Overview](ForceTerminateConnectionAPI/force_terminate_connection_api.html) - [API Reference](ForceTerminateConnectionAPI/force_terminate_connection_api_reference.html) - RFID Management API - [Overview](RFIDManagementAPI/rfid_management_api.html) - [API Reference](RFIDManagementAPI/rfid_management_api_reference.html) - Remote Commands API - [Overview](RemoteCommandAPI/remote_command_api.html) - API Reference - [Version 2](RemoteCommandAPI/remote_command_api_reference_v2.html) - [Version 1](RemoteCommandAPI/remote_command_api_reference.html) - Reservations API - [Overview](ReservationsAPI/reservations_api.html) - [API Reference](ReservationsAPI/reservations_api_reference.html) - Charger Control API - [Overview](ChargerControlAPI/charger_control_api.html) - [API Reference](ChargerControlAPI/charger_control_api_reference.html) - Virtual Power Plant API - [Overview](VPPAPI/vpp_api.html) - [API Reference](VPPAPI/vpp_api_reference.html) - VDV-261 - [Overview](VDV261/vdv261_api.html) - [API Reference](VDV261/vdv261_api_reference.html) - [Contact](contact.html) # DepotFinity Developer APIs [DepotFinity](https://www.siemens.com/global/en/products/energy/medium-voltage/solutions/emobility/ebus-depot/evdepot-digital.html) is a cloud-based software service that is used to monitor, report, schedule and manage the charging operations of chargers within a depot. DepotFinity API documentation aims to describe the specifications, conventions, steps to get started, and how to use DepotFinity APIs. ## DepotFinity APIs allow you to perform the following functions - Manage tokens - Manage transactions - Control charging power limits - Generate session reports - Get energy prices and power consumption - Send remote commands to the charger - Terminate an active charger connection - Interface with EV using VDV-261 parameters *(For more information see VDV-261)* # Charger Control API The Charger Control API allows you to perform the following functions. - Get list of power groups from external systems. - Update the external power limit of a charger group from external systems. - Prioritize and allocate the power sharing across different loads dynamically. ## Prerequisites Request for the Charger Group ID list from Siemens Regional Support. ## API Usage See [Token Management API](../TokenManagementAPI/token_management_api.html) for more information on how to create and use the access token. ## External Power Limit ### Prepare Request Body To prepare a request body, perform the following steps. 1. Set charger group GUID to the request path parameter `chargerGroupId`. 1. Set `Content-Type` header as `application/json` and use the parameters schema `ExternalPowerLimit` as described in [Charger Control API Reference](charger_control_api_reference.html). #### Example Request Note Use the server URL applicable to your region. See [API Reference](charger_control_api_reference.html) for available server URLs applicable to your region. ```bash export ACCESS_TOKEN = export SOURCE = export EXTERNALPOWERRATEUNIT = export FROMDATE = # Example "2022-11-22T19:54:10.143Z" export TODATE = # Example "2022-11-23T22:18:09.797Z" export EXTERNALPOWERLIMIT = export CHARGER_GROUP_ID = curl --location --request PUT 'https://api.eu.depot.emobility.io/v1/chargerControl/chargerGroups/$CHARGER_GROUP_ID/externalPowerLimits' \ --header 'Content-Type: application/json' \ --header 'Accept: application/json' \ --header 'Authorization: Bearer $ACCESS_TOKEN' \ --data-raw '{ "source": "$SOURCE", "externalPowerRateUnit": "$EXTERNALPOWERRATEUNIT", "dynamicPowerLimit": [ { "fromDate": "$FROMDATE", "toDate": "$TODATE", "externalPowerLimit": $EXTERNALPOWERLIMIT } ] }' ``` ### Response Body Successful response returns 200 HTTP status code with response body as `Success` schema. For more information, see [Charger Control API Reference](charger_control_api_reference.html). #### Example Response ```json { "message": "External power limit saved successfully." } ``` ## Power Groups ### Prepare Request Body To prepare a request body, perform the following steps. 1. Set `Content-Type` header as `application/json` as described in [Charger Control API Reference](charger_control_api_reference.html). #### Example Request Note Use the server URL applicable to your region. See [API Reference](charger_control_api_reference.html) for available server URLs applicable to your region. ```bash export ACCESS_TOKEN = curl --location --request GET 'https://api.eu.depot.emobility.io/v1/chargerGroup/powergroups' \ --header 'Content-Type: application/json' \ --header 'Authorization: Bearer $ACCESS_TOKEN' ``` ### Response Body Successful response returns 200 HTTP status code with response body as `PowerGroup` schema. For more information, see [Charger Control API Reference](charger_control_api_reference.html). #### Example Response ```json [ { "PowerGroupID": 0123, "CommonName": "xyz", "Algorithm": "FO", "DefaultMaxPower": 200000, "DefaultPrice": 0, "CreatedOn": "2022-05-13T03:00:35.830992", "CreatedBy": "john@example.com", "ModifiedOn": "2023-06-05T11:59:51", "ModifiedBy": "henry@example.com", "FullUpdateRequired": true, "ChargeStrategy": null, "IsOptimalChargingActive": false, "LastAlgorithmTriggeredTime": null, "LastAlgorithmRetriedTime": null, "PowerElementEntries": [ { "DefaultProfileStatus": "Pending", "Type": "Charger", "PowerElementID": "abc123", "DefaultProfileRetryCount": 0, "SendClearProfile": false }, { "DefaultProfileStatus": "Rejected", "Type": "Charger", "PowerElementID": "qwerty", "DefaultProfileRetryCount": 3, "SendClearProfile": false } ], "SetLimitsRetryNecessary": false, "LastPowerLimits": { "ClientID": "PQRS", "GUID": "f29d2af3-6633-0000-920d-eb51b21f0000", "Result": [ { "ChargerID": "xyz", "Connectors": [ { "ChargingZone": "charging", "ConnectorID": 1, "ETC": 0, "PowerSetPoint": 12000 } ] } ] }, "PowerSchedulingEntries": [], "UserDefaultPowerProfile": "ChargerMaxPower", "PriceSchedulingEntries": [], "ID": "025052e4-1111-2222-abd4-22a01f1123456" } ] ``` # Charger Control API # Charger Status API The Charger Status API allows you to obtain the following information. - *Charger availability status*: charger is `Available`, `Unavailable`, and `Faulted`. - *Connector availability status*: connector is `Available`, `Unavailable`, and so on. For the different connector availability statuses, see [Charger Status API Reference](charger_status_api_reference.html). - *Charger connectivity status*: charger is connected or not connected. ## API Usage See [Token Management API](../TokenManagementAPI/token_management_api.html) for more information on how to create and use the access token. ## Charger Availability Status ### Prepare Request Body To prepare a request body, perform the following steps. 1. Set charger ID to the request path parameter `chargerId`. See [Charger Status API Reference](charger_status_api_reference.html). #### Example Request Note Use the server URL applicable to your region. See [API Reference](charger_status_api_reference.html) for available server URLs applicable to your region. ```bash export ACCESS_TOKEN = export CHARGER_ID = curl --location --request GET 'https://api.eu.depot.emobility.io/v1/chargerStatus/chargers/$CHARGER_ID/availabilityStatus' \ --header 'Accept: application/json' \ --header 'Authorization: Bearer $ACCESS_TOKEN' ``` ### Response Body Successful response returns 200 HTTP status code with response body as `ChargerAvailabilityStatusList` schema. For more information, see [Charger Status API Reference](charger_status_api_reference.html). #### Example Response ```json { "items": [ { "timeStamp": "2022-10-18T08:37:44.210Z", "errorCode": "NoError", "vendorErrorCode": 1, "vendorId": 23, "status": "Available" } ] } ``` ## Connector Availability Status ### Prepare Request Body To prepare a request body, perform the following steps. 1. Set charger ID to the request path parameter `chargerId`. 1. Set connector ID to the request path parameter `connectorId`. See [Charger Status API Reference](charger_status_api_reference.html). #### Example Request Note Use the server URL applicable to your region. See [API Reference](charger_status_api_reference.html) for available server URLs applicable to your region. ```bash export ACCESS_TOKEN = export CHARGER_ID = export CONNECTOR_ID = curl --location --request GET 'https://api.eu.depot.emobility.io/v1/chargerStatus/chargers/$CHARGER_ID/connectors/$CONNECTOR_ID/availabilityStatus' \ --header 'Accept: application/json' \ --header 'Authorization: Bearer $ACCESS_TOKEN' ``` ### Response Body Successful response returns 200 HTTP status code with response body as `ChargerAvailabilityStatusList` schema. For more information, see [Charger Status API Reference](charger_status_api_reference.html). #### Example Response ```json { "items": [ { "timeStamp": "2022-10-18T08:37:44.210Z", "errorCode": "NoError", "vendorErrorCode": 23, "vendorId": 1, "status": "Available" } ] } ``` ## Charger Connectivity Status ### Prepare Request Body To prepare a request body, perform the following steps. 1. Set charger ID to the request path parameter `chargerId`. See [Charger Status API Reference](charger_status_api_reference.html). #### Example Request Note Use the server URL applicable to your region. See [API Reference](charger_status_api_reference.html) for available server URLs applicable to your region. ```bash export ACCESS_TOKEN = export CHARGER_ID = curl --location --request GET 'https://api.eu.depot.emobility.io/v1/chargerStatus/chargers/$CHARGER_ID/connectivityStatus' \ --header 'Accept: application/json' --header 'Authorization: Bearer $ACCESS_TOKEN' ``` ### Response Body Successful response returns 200 HTTP status code with response body as `ChargerConnectivityStatusList` schema. For more information, see [Charger Status API Reference](charger_status_api_reference.html). #### Example Response ```json { "items": [ { "time": "2022-08-18T05:23:04.334Z", "value": "true" } ] } ``` # Charger Status API # Force Termination API The Force Termination API allows you to terminate the charger connection and make it offline. ## API Usage See [Token Management API](../TokenManagementAPI/token_management_api.html) for more information on how to create and use the access token. ## Force Terminate Connection ### Prepare Request Body To prepare a request body, perform the following steps. 1. Set charger ID to the request path parameter `chargerId`. 1. Set `Content-Type` header as `application/json` as described in [Force Termination API Reference](force_terminate_connection_api_reference.html). #### Example Request Note Use the server URL applicable to your region. See [API Reference](force_terminate_connection_api_reference.html) for available server URLs applicable to your region. ```bash export ACCESS_TOKEN = export CHARGER_ID = curl --location --request PUT 'https://api.eu.depot.emobility.io/v1/chargerControl/$CHARGER_ID/forceTerminateConnection' \ --header 'Content-Type: application/json' \ --header 'Authorization: Bearer $ACCESS_TOKEN' ``` ### Response Body Successful response returns 200 HTTP status code with response body as `Success` schema. For more information, see [Force Termination API Reference](force_terminate_connection_api_reference.html). #### Example Response ```json { "code": 200, "message": "Charger Accepted." } ``` # Force Termination API # RFID Management API RFID Management API allows users to add and manage RFID cards. Enabling RFID authentication will require you to present your RFID card to start and stop charging sessions. You can also onboard multiple RFID cards in a single go. ## API Usage You must be authorized to use RFID Management API. See [Token Management API](../TokenManagementAPI/token_management_api.html) for more information on how to create and use the access token. ## Bulk Import RFIDs This method allows you to insert or upload Bulk RFIDs. ### Prepare Request You can prepare the request body with parameter `interimVehicleIDs` in an array format. For more information, see [RFID API Reference](rfid_management_api_reference.html). ### Request Body ```json { "interimVehicleIDs": ["YOUR_RFID"] } ``` #### Example Request Note Use the server URL applicable to your region. See [API Reference](rfid_management_api_reference.html) for available server URLs applicable to your region. ```bash export TOKEN = export INTERIMVEHICLEIDS = [] curl --location --request POST 'https://api.eu.depot.emobility.io/v2/rfid' \ --header 'Content-Type: application/json' \ --header 'Accept: application/json' \ --header 'Authorization: Bearer $TOKEN' \ --data '{ "interimVehicleIDs": $INTERIMVEHICLEIDS }' ``` ### Response Body Successful response returns 201 HTTP status code with response body of `RFIDImportResponse` schema. For more information, see [RFID API Reference](rfid_management_api_reference.html). #### Example Response ```json { "message": "RFID has been created successfully." } ``` ## Status Update of a RFID This method allows you to enable or disable RFID authentication. ### Prepare Request You can prepare the request with path parameter `TAG` is RFID and `STATUS` is boolean format. For more information, see [RFID API Reference](rfid_management_api_reference.html). #### Example Request ```bash export TOKEN = export TAG = export STATUS = curl --location --request POST 'https://api.eu.depot.emobility.io/v2/rfid/$TAG/$STATUS' \ --header 'Content-Type: application/json' \ --header 'Accept: application/json' \ --header 'Authorization: Bearer $TOKEN' ``` ### Response Body Successful response returns 201 HTTP status code with response body of `RFIDUpdateResponse` schema. For more information, see [RFID API Reference](rfid_management_api_reference.html). #### Example Response ```json { "message": "RFID status has been updated successfully." } ``` # RFID Management API # Remote Command API The Remote Command API allows you send commands to a remote charger and perform the following functions. - Control charging activity - Modify charger settings and configurations ## API Usage See [Token Management API](../TokenManagementAPI/token_management_api.html) for more information on how to create and use the access token. ## Remote Command ### Prepare Request Body Set the following path parameters to prepare the request body. 1. Charger ID to `chargerId`. 1. Remote Command Type to `remoteCommandType`. 1. Additional body parameter for sending the payload. Refer the schema `RemoteCommandBody` for the additional body parameter details. For more information, see [Remote Command API Reference](remote_command_api_reference.html). #### Example Request Note Use the server URL applicable to your region. See [API Reference](remote_command_api_reference.html) for available server URLs applicable to your region. ```bash export ACCESS_TOKEN = export CHARGER_ID = export REMOTE_COMMAND_TYPE = export CONNECTOR_ID = export ID_TAG = curl --location --request POST 'https://api.eu.depot.emobility.io/v1/chargerControl/chargers/$CHARGER_ID/remoteCommands/$REMOTE_COMMAND_TYPE' --header 'Content-Type: application/json' \ --header 'Authorization: Bearer $ACCESS_TOKEN' \ --data '{ "connectorId": $CONNECTOR_ID, "idTag": $ID_TAG }' ``` ### Response Body Successful response returns 200 HTTP status code with response body as `RemoteCommandResponse` schema. For more information, see [Remote Command API Reference](remote_command_api_reference.html). #### Example Response ```json { "commandType": "startChargingSession", "response": { "status": "Accepted", "message": "Operation successful: Start charging request has been accepted." } } ``` ### Example Responses | Http Code | Status | Message | | --------- | -------- | --------------------------------------------------------------- | | 200 | Accepted | Operation successful: Start charging request has been accepted. | | 200 | Accepted | Operation successful: Stop charging request has been accepted. | | 400 | Rejected | Operation denied: Rejected by the charger. | | 400 | | Operation denied: Invalid Payload. | | 404 | | Operation denied: Charger is offline. | | 404 | | Operation denied: Charger ID not found. | | 404 | | Operation denied: Connector ID not found. | | 404 | | Operation denied: No ongoing charging. | | 401 | | Operation denied: Unauthorized IDTag. | | 500 | | Operation failed: Internal server error. | | 408 | | Operation failed: Timeout. | # Remote Command API # Remote Command API v2 # Reservations API Reservations API allow users to manage connector reservations for charging. ## API Usage You must be authorized to use Reservations API. See [Token Management API](../TokenManagementAPI/token_management_api.html) for more information on how to create and use the access token. ## Add Reservation This method allows you to add a reservation. ### Prepare Request To prepare a request body, perform the following steps. 1. Set Charger ID to the request path parameter `chargerId`. 1. Set Connector ID to the request path parameter `connectorId`. 1. Set Charging Session Start time to the request body parameter `ReservationStart`. 1. Set Charging Session End time to the request body parameter `ReservationEnd`. 1. Set mode of authorization to the request body parameter `AuthorizationType`. 1. Set authorization id value which is RFID Tag Number in this case, to the request body parameter `AuthorizationValue`. ### Sample Request Body ```json { "ReservationStart": "2024-01-01T08:30:19.220Z", "ReservationEnd": "2024-01-01T11:30:19.220Z", "AuthorizationType": "RFID", "AuthorizationValue": "89C4704A" } ``` #### Example Request Note Use the server URL applicable to your region. See [API Reference](reservations_api.html) for available server URLs applicable to your region. ```bash export ACCESS_TOKEN = export CHARGER_ID = export CONNECTOR_ID = export RESERVATION_START = # Example "2022-11-22T19:54:10.143Z" export RESERVATION_END = # Example "2022-11-22T21:54:10.143Z" export AUTHORIZATION_TYPE = # Example "RFID" export AUTHORIZATION_VALUE = # Example "59170C5D" curl --location --request POST 'https://api.eu.depot.emobility.io/v1/reservations/chargers/$CHARGER_ID/connectors/$CONNECTOR_ID' \ --header 'Content-Type: application/json' \ --header 'Accept: application/json' \ --header 'Authorization: Bearer $ACCESS_TOKEN' \ --data '{ "ReservationStart": $RESERVATION_START, "ReservationEnd": $RESERVATION_END, "AuthorizationType": $AUTHORIZATION_TYPE, "AuthorizationValue": $AUTHORIZATION_VALUE, }' ``` ### Response Body Successful response returns 201 HTTP status code with response body of `ReservationSuccessResponse` schema. #### Example Response ```json { "ChargerReservationID": "f47ac10b-58cc-4372-a567-0e02b2c3d479", "message": "Charger Reservation is successful." } ``` ## Remove Reservation This method allows you to remove booking based on reservation ID. ### Prepare Request You can prepare the request with path parameter `reservationId` which was received in the response at the time of booking. Note Use the server URL applicable to your region. See [API Reference](reservations_api.html) for available server URLs applicable to your region. #### Example Request ```bash export ACCESS_TOKEN = export RESERVATION_ID = curl --location --request DELETE 'https://api.eu.depot.emobility.io/v1/reservations/$RESERVATION_ID' \ --header 'Content-Type: application/json' \ --header 'Authorization: Bearer $ACCESS_TOKEN' ``` ### Response Body Successful response returns 201 HTTP status code with response body of `ReservationRemovedSuccessResponse` schema. #### Example Response ```json { "code": 200, "message": "Reservation is deleted successful." } ``` For more information, see [Reservations API Reference](reservations_api_reference.html). # Reservations API # Session Reports API Session Report API allows you to read the session report information of HPC chargers with legacy CCBox in raw format. The query response contains an array of session report objects. To limit the query results for a selected time period or number of result items, use `from`, `to`, `page`, and `limit` parameters in your request body. ## API Usage See [Token Management API](../TokenManagementAPI/token_management_api.html) for more information on how to create and use the access token. ## Session Reports ### Prepare Request Body Set the following query parameters to prepare the request body. 1. Charger ID to `chargerId`. 1. Connector ID to `connectorId`. 1. Additional query parameter for filtering the response. For more information, see [Session Reports API Reference](session_reports_api_reference.html). #### Example Request Note Use the server URL applicable to your region. See [API Reference](session_reports_api_reference.html) for available server URLs applicable to your region. ```bash export ACCESS_TOKEN = export CHARGER_ID = export CONNECTOR_ID = curl --location --request GET 'https://api.eu.depot.emobility.io/v1/sessionReports?chargerId=$CHARGER_ID&connectorId=$CONNECTOR_ID' --header 'Authorization: Bearer $ACCESS_TOKEN' ``` ### Response Body Successful response returns 200 HTTP status code with response body as `SessionReports` schema. For more information, see [Session Reports API Reference](session_reports_api_reference.html). #### Example Response ```json { "items": [ { "sessionId": 123456789, "vehicleId": "qwerty", "ocppStatus": [ { "value": "Available", "eventTimestamp": "2021-03-23T16:54:56.087Z" }, { "value": "Preparing", "eventTimestamp": "2021-03-23T16:59:11.670Z" }, ... ], "detailStatus": [ { "key": "systemStatus", "value": "0x00000000", "eventTimestamp": "2021-03-23T00:48:26.437Z" }, { "key": "operationalStatus", "value": "0x00000000", "eventTimestamp": "2021-03-23T00:48:26.437Z" }, ... ], "deviceStatus": [ { "status": "OPEN", "deviceId": "Breaker", "eventTimestamp": "2021-03-23T16:52:33.173Z" }, { "status": "STANDBY", "deviceId": "Converter", "errorCodes": [ { "errorCode": "WARN: converter : sibcos 1: VQS disabled from operating mode(no:144)", "errorCounter": 29 }, ... ], "eventTimestamp": "2021-03-23T16:55:11.088Z" } ], "energySocEnd": 98, "energySocStart": 76, "sessionTimeEnd": "2022-08-03T16:40:30.440Z", "energyChargeEnd": 1046400, "chargeParameters": [ { "maxPower": 450000, "paramType": "Station", "maxCurrent": 600, "maxVoltage": 750, "minCurrent": 0, "minVoltage": 400, "eventTimestamp": "2021-03-23T16:59:12.052Z" }, { "maxPower": 600000, "paramType": "Request", "maxCurrent": 800, "maxVoltage": 750, "minCurrent": 0, "minVoltage": 0, "eventTimestamp": "2021-03-23T16:59:12.186Z" }, ... ], "energyPrimaryEnd": 293138710, "sessionTimeStart": "2022-08-03T16:20:30.661Z", "energyChargeStart": 1009136, "energyPrimaryStart": 293099449, "energyRequestedEnd": 1054122, "sessionMeterValues": [ { "key": "present_current", "value": 0, "eventTimestamp": "2021-03-23T16:54:35.287Z" }, { "key": "present_voltage", "value": 3, "eventTimestamp": "2021-03-23T16:59:08.685Z" }, ... ], "energyRequestedStart": 1016608, "batteryAlignmentResults": { "inRange": true, "eventTimestamp": "2021-03-23T16:59:38.672Z", "presentVoltage": 633, "requestedVoltage": 633 }, "dcBreakerOpenConditions": { "current": 0, "voltage": 676, "eventTimestamp": "2021-03-23T17:13:44.735Z" } } ], "ChargerID": "xyz", "ConnectorID": "1" } ``` # Session Reports API # Time Series API The Time Series API allows you to fetch the time series data records of a charger or the connector of a charger. You can query with `measurand` and optionally provide `phase` values to query the time series data. If the duration for which to fetch the data records are not specified using the `from` and `to` parameters, only the previous day records are retrieved in the query results. ## API Usage See [Token Management API](../TokenManagementAPI/token_management_api.html) for more information on how to create and use the access token. ## Time Series of Charger ### Prepare Request Body To prepare a request body, perform the following steps. 1. Set charger ID to the request path parameter `chargerId`. See [Time Series API Reference](time_series_api_reference.html). #### Example Request Note Use the server URL applicable to your region. See [API Reference](time_series_api_reference.html) for available server URLs applicable to your region. ```bash export ACCESS_TOKEN = export CHARGER_ID = export MEASURAND = export PHASE = curl --location --request GET 'https://api.eu.depot.emobility.io/v1/timeSeries/chargers/$CHARGER_ID?measurand=$MEASURAND&phase=$PHASE' \ --header 'Accept: application/json' \ --header 'Authorization: Bearer $ACCESS_TOKEN' ``` ### Response Body Successful response returns 200 HTTP status code with response body as `TimeSeriesDataList` schema. For more information, see [Time Series API Reference](time_series_api_reference.html). #### Example Response ```json { "items": [ { "timeStamp": "2022-08-18T05:23:04.334Z", "value": "true" } ] } ``` #### Measurand and Phase Mapping for Charger | Measurand | Phase | | ------------------ | ---------------------------- | | ActivePowerExport | NA | | ActivePowerImport | NA | | PowerOffered | NA | | ActiveEnergyExport | NA | | ActiveEnergyImport | NA | | CurrentExport | L1, L2, L3 | | CurrentImport | L1, L2, L3 | | CurrentOffered | L1, L2, L3 | | Voltage | L1, L2, L3, L1-N, L2-N, L3-N | | AlarmCode | NA | | Temperature | NA | ## Time Series of Connector ### Prepare Request Body To prepare a request body, perform the following steps. 1. Set charger ID to the request path parameter `chargerId`. 1. Set connector ID to the request path parameter `connectorId`. See [Time Series API Reference](time_series_api_reference.html). #### Example Request Note Use the server URL applicable to your region. See [API Reference](time_series_api_reference.html) for available server URLs applicable to your region. ```bash export ACCESS_TOKEN = export CHARGER_ID = export CONNECTOR_ID = export MEASURAND = curl --location --request GET 'https://api.eu.depot.emobility.io/v1/timeSeries/chargers/$CHARGER_ID/connectors/$CONNECTOR_ID?measurand=$MEASURAND' \ --header 'Accept: application/json' \ --header 'Authorization: Bearer $ACCESS_TOKEN' ``` ### Response Body Successful response returns 200 HTTP status code with response body as `TimeSeriesDataList` schema. For more information, see [Time Series API Reference](time_series_api_reference.html). #### Example Response ```json { "items": [ { "timeStamp": "2022-08-18T05:23:04.334Z", "value": "true" } ] } ``` #### Measurand and Phase Mapping for Connector | Measurand | Phase | | ------------------ | ------------------------------------- | | ActivePowerExport | NA | | ActivePowerImport | NA | | PowerOffered | NA | | ActiveEnergyExport | NA | | ActiveEnergyImport | NA | | CurrentExport | L1, L2, L3 | | CurrentImport | L1, L2, L3 | | CurrentOffered | L1, L2, L3 | | Voltage | L1-N, L2-N, L3-N, L1-L2, L2-L3, L3-L1 | | SoC | NA | | AlarmCode | NA | | Temperature | NA | | PowerFactor | NA | # Time Series API # Token Management API The token management API allows your application to make requests on your behalf and access data from the authorized services. For every valid request to the authorization server, an access token response for the client application is issued with the following properties. - `access_token`: Access token string issued by the authorization server. - `expires_in`: The time in seconds for which the access token is valid. - `token_type`: The type of token, typically just the string "bearer". Note Your token is valid for a limited time and your application will need to refresh it at regular intervals. If the access token request is invalid, error responses are returned with the HTTP 403 status code. For detailed information on the error responses, see [General Information](../general_information.html#http-status-codes). ## Create a Token Create a JWT (JSON Web Token) request with the Authorization header format for Token Management API. The following sample shows the format of the Authorization header. ```ts const CLIENT_ID=; const CLIENT_SECRET=; // create a base64 encoded string with CLIENT_ID and CLIENT_SECRET const YOUR_TOKEN = Base64Encoder("${CLIENT_ID}:${CLIENT_SECRET}"); // add the authorization header Authorization: Basic ${YOUR_TOKEN} ``` To run the following example, set your `TOKEN`. ## Example Requests Note Use the server URL applicable to your region. See [API Reference](token_management_api_reference.html) for available server URLs applicable to your region. ```bash export TOKEN = curl --location --request POST 'https://clientapp-auth.eu.depot.emobility.io/oauth2/token' \ --header 'Accept: application/json' \ --header 'Content-Type: application/x-www-form-urlencoded' \ --header 'Authorization: Basic $TOKEN' \ --data-urlencode 'grant_type=client_credentials' ``` ## Example Response The `access_token` property in the response body is returned after the successful execution of the request. You can now pass the `access_token` in the `Authorization` header of any subsequent API requests. The `expires_in` property represents the number of seconds your token is valid. The token is valid for 24 hours (86400 seconds) by default. When this time has elapsed you will need to create a new token. ```json { "access_token": "eyJ0eXAiOiUSJ9.eyJpc3MiOiJdGlhbHMifQ.MJpcxLfyOt", "expires_in": 3600, "token_type": "Bearer" } ``` Note Your token is valid for a limited time and your application will need to refresh it at regular intervals. Warning Make sure to store your token in a safe place, never share it or store it in source control. Anyone with access to your token can access the system on your behalf. ## Token Usage Pass the `access_token` from the token API in the header to access DepotFinity APIs. The following sample shows the format of the Authorization header. ```bash export ACCESS_TOKEN = # add the authorization header Authorization: Bearer ${ACCESS_TOKEN} ``` # Token Management API # Transaction Management API A transaction is a charging event that occurs at a specific time for a specific charger on a specific connector. Each transaction is represented by an unique identifier called `TransactionID`. The Transaction Management API allows you to read the transaction details for a selected time duration. The transaction details can also be retrieved for a selected `chargerId`, `connectorId`, `vehicleId`, or `transactionID`. All responses are received in JSON format. The query responses are shown as a list. You can specify the number of list items shown in a page. By default, 100 items are displayed in a single page. The results can be sorted by `TransactionID`, `StartAt`, and `EndAt` of the transaction. ## API Usage You must be authorized to use Transaction Management API. See [Token Management API](../TokenManagementAPI/token_management_api.html) for more information on how to create and use the access token. ## Get All Transaction This method allows you to fetch all the transaction details for a selected time duration. ### Prepare Request You can query with various parameters like `from`, `to`, `order`, and `limit`. For more information, see [Transactions API Reference](transaction_management_api_reference.html). #### Example Request Note Use the server URL applicable to your region. See [API Reference](transaction_management_api_reference.html) for available server URLs applicable to your region. ```bash export TOKEN = export CHARGER_ID = export CONNECTOR_ID = export VEHICLE_ID = export ORDER_BY = export ORDER = export PAGE = export LIMIT = curl --location --request GET 'https://api.eu.depot.emobility.io/v2/transactions?chargerId=$CHARGER_ID&connectorId=$CONNECTOR_ID&vehicleId=$VEHICLE_ID&orderBy=$ORDER_BY&order=$ORDER&page=$PAGE&limit=$LIMTIT' \ --header 'Accept: application/json' \ --header 'Authorization: Bearer $TOKEN' ``` ### Response Body Successful response returns 200 HTTP status code with response body as array of `Transaction` schema. For more information, see [Transactions API Reference](transaction_management_api_reference.html). #### Example Response ```json [ { "TransactionID": 010203, "ChargerID": "xyz", "ConnectorID": 1, "StartAt": "2022-10-27T08:52:34.000Z", "EndAt": "2022-10-27T08:53:07.000Z", "InitialSoC": 0, "VehicleID": "0123", "Initiator": "0123", "EndSoC": 0, "EnergyDelivered": 0, "EndResult": "Local", "ExtraInfo": { "Initiator": "0123" }, "SessionStartTime": null, "SessionEndTime": null, "InsertedAt": "2022-10-27T08:53:07.000Z" }, { "TransactionID": 010204, "ChargerID": "xyz", "ConnectorID": 1, "StartAt": "2022-10-27T06:15:11.000Z", "EndAt": "2022-10-27T06:51:34.000Z", "InitialSoC": 32, "VehicleID": "0123", "Initiator": "0123", "EndSoC": 100, "EnergyDelivered": 12833, "EndResult": "Incomplete", "ExtraInfo": { "Initiator": "0123" }, "SessionStartTime": null, "SessionEndTime": null, "InsertedAt": "2022-10-27T06:51:34.000Z" }, ... ] ``` If you are using this API to fetch the transactions periodically for a relative time frame (e.g. fetching transactions for the last 24-hours), then it is recommended to use filter by InsertedAt. This is to ensure that offline transactions that were reported to DepotFinity after the relative time frame, are also included in the response. ### Example Scenario: A daily run fetches transactions periodically for the last 24 hours, Initially Charger is online - 03-05-2023 10:00 AM: - Transaction 1 started - 03-05-2023 10:30 AM: - Transaction 1 stopped - 03-05-2023 11:00 AM: - Charger went offline - 03-05-2023 02:00 PM: - Transaction 2 started - 03-05-2023 02:30 PM: - Transaction 2 stopped - 04-05-2023 06:00 AM: - Daily run `filterBy: StartAt` fetches (Transaction 1) - 04-05-2023 10:30 AM: - Transaction 3 started - 04-05-2023 11:00 AM: - Transaction 3 stopped - 04-05-2023 12:00 PM: - Charger Online - 05-05-2023 06:00 AM: - Daily run `filterBy: StartAt` fetches (No Transactions) - 05-05-2023 06:10 AM: - Daily run with `filterBy: InsertedAt` fetches transactions that were reported in offline (Transaction 2, Transaction 3) ## Get Specific Transaction This method allows you to fetch the details for a specific transaction. ### Prepare Request Set the request path parameter `TransactionID` for the transaction to be retrieved. For more information, see [Transactions API Reference](transaction_management_api_reference.html). #### Example Request Note Use the server URL applicable to your region. See [API Reference](transaction_management_api.html) for available server URLs applicable to your region. ```bash export ACCESS_TOKEN = export TRANSACTION_ID = curl --location --request GET 'https://api.eu.depot.emobility.io/v2/transactions/$TRANSACTION_ID' \ --header 'Accept: application/json' \ --header 'Authorization: Bearer "$ACCESS_TOKEN' ``` ### Response Body Successful response returns 200 HTTP status code with response body as `Transaction` schema. For more information, see [Transactions API Reference](transaction_management_api_reference.html). #### Example Response ```json { "TransactionID": 010203, "ChargerID": "abcd", "ConnectorID": 1, "StartAt": "2022-11-18T04:39:02.000Z", "EndAt": "2022-11-18T04:40:12.000Z", "InitialSoC": 59, "VehicleID": "9876", "Initiator": "9876", "EndSoC": 0, "EnergyDelivered": 0, "EndResult": "Incomplete", "ExtraInfo": { "Initiator": "9876" }, "SessionStartTime": null, "SessionEndTime": null, "InsertedAt": "2022-11-18T04:40:12.000Z" } ``` # Transaction Management API # VDV-261 API The VDV-261 API facilitates an interface that allows the EV (Electric Vehicle) to communicate and send VDV-261 parameters to the DepotFinity backend. Software solutions serve as a uniform communication means between the EV and different backend systems. The communication between the EV and charging station is a part of the ISO standard 15118, Ed. 1 which is now an established standard. ## Prerequisites Complete the following prerequisite actions. 1. Enable preconditioning after onboarding an EV. 1. Enable secure connection to the EV and copy it's credentials. 1. Request for V2ICP root certificate and credentials from Siemens Regional Support. 1. Schedule the vehicle on smart charging group and confirm that the password is synchronized. 1. Execute the VDV-261 API with basic authentication and necessary body parameters. ## API Usage The VDV-261 API uses basic authentication method to authenticate the EV. DepotFinity uses the "VIN" (Vehicle Identification Number) as `vin` and `password`. The following sample shows the format of the Authorization header. ```ts const VIN = ; const PASSWORD = ; // create a base64 encoded string with VIN and PASSWORD const YOUR_VDV261_TOKEN = Base64Encoder("${VIN}:${PASSWORD}"); // add the authorization header Authorization: Basic ${YOUR_VDV261_TOKEN} ``` ## Prepare Request Body To prepare a request body, set the `Content-Type` header as `application/json` and use the parameters as described in [VDV-261 API Reference](vdv261_api_reference.html). ### Example Request Note Use the server URL applicable to your region. See [API Reference](vdv261_api_reference.html) for available server URLs applicable to your region. ```bash export VDV261_TOKEN = export SEQ = , export VIN = , export EVCCID = export ODO = export BAT_REQTIME = export BAT_EAMOUNT = export BPREC_EAMOUNT = export PREC_REQTIME = export PREC_EAMOUNT = export CHRG_STAT = curl --location --request POST 'https://ev.eu.depot.emobility.io/vdv' \ --header 'Content-Type: application/json' \ --header 'Accept: application/json' \ --header 'Authorization: Basic $VDV261_TOKEN' \ --data-raw '{ "seq": "$SEQ", "vin": "$VIN", "evccid": "$EVCCID", "odo": $ODO, "bat_reqtime": $BAT_REQTIME, "bat_eamount": $BAT_EAMOUNT, "bprec_eamount": $BPREC_EAMOUNT, "prec_reqtime": $PREC_REQTIME, "prec_eamount": $PREC_EAMOUNT, "chrg_stat": $CHRG_STAT }' ``` ## Response Body Successful response returns 200 HTTP status code with response body as `TimeSeriesResponseBody` schema. For more information, see [VDV-261 API Reference](vdv261_api_reference.html). ### Example Response On the first request with VIN and other parameters, the response contains `seq`, `driveoff`, `prec_dsrd`, `prec_hvac`, `ambienttemp`, and `vin` parameters. ```json { "seq": 0, "driveoff": 409, "prec_dsrd": 1, "prec_hvac": 3, "ambienttemp": 255, "vin": "120" } ``` On subsequent requests with the same parameters, the response contains only `vin` and `seq` parameters. ```json { "seq": 0, "vin": "120" } ``` # VDV-261 API # Virtual Power Plant API Virtual Power Plant (VPP) integration API allows you to get and update energy price information that can be used to optimize services and power consumption. ## API Usage See [Token Management API](../TokenManagementAPI/token_management_api.html) for more information on how to create and use the access token. ## Update Energy Prices ### Prepare Request Body To prepare a request body, perform the following steps. 1. Set site ID to the request path parameter `siteId`. 1. Set `Content-Type` header as `application/json` and use the parameters schema `VPPEnergyPricesBody` as described in [VPP API Reference](vpp_api_reference.html). #### Example Request Note Use the server URL applicable to your region. See [API Reference](vpp_api_reference.html) for available server URLs applicable to your region. Note Make sure to enter the currency values in the same currency type always. ```bash export ACCESS_TOKEN = export SITE_ID = export START = export DURATION = export PRICE = curl --location --request POST 'https://api.eu.depot.emobility.io/v1/vppController/$SITE_ID/energyPrices' \ --header 'Content-Type: application/json' \ --header 'Accept: application/json' \ --header 'Authorization: Bearer $ACCESS_TOKEN' \ --data-raw '{ "PricesJsonStr": { "blocks": [ { "start": $START, "duration": $DURATION, "price": $PRICE } ] } }' ``` ### Response Body Successful response returns 200 HTTP status code with response body as `Success` schema. For more information, see [VPP API Reference](vpp_api_reference.html). #### Example Response ```json { "code": 200, "message": "Energy prices have been updated." } ``` ## Get Energy Prices ### Prepare Request Body To prepare a request body, perform the following steps. 1. Set site ID to the request path parameter `siteId`. See [VPP API Reference](vpp_api_reference.html). #### Example Request Note Use the server URL applicable to your region. See [API Reference](vpp_api_reference.html) for available server URLs applicable to your region. ```bash export ACCESS_TOKEN = export SITE_ID = curl --location --request GET 'https://api.eu.depot.emobility.io/v1/vppController/$SITE_ID/energyPrices' \ --header 'Accept: application/json' \ --header 'Authorization: Bearer $ACCESS_TOKEN' ``` ### Response Body Successful response returns 200 HTTP status code with response body as `VPPEnergyPricesResponse` schema. For more information, see [VPP API Reference](vpp_api_reference.html). #### Example Response ```json { "ClientID": "XYZ", "SiteID": "ABC", "PricesJsonStr": { "blocks": [ { "start": "2023-02-04T23:00:00Z", "duration": 30, "price": 9.74 } ] } } ``` ## Get Power Consumption ### Prepare Request Body To prepare a request body, perform the following steps. 1. Set site ID to the request path parameter `siteId`. See [VPP API Reference](vpp_api_reference.html). #### Example Request Note Use the server URL applicable to your region. See [API Reference](vpp_api_reference.html) for available server URLs applicable to your region. ```bash export ACCESS_TOKEN = export SITE_ID = curl --location --request GET 'https://api.eu.depot.emobility.io/v1/vppController/$SITE_ID/powerConsumption' \ --header 'Accept: application/json' \ --header 'Authorization: Bearer $ACCESS_TOKEN' ``` ### Response Body Successful response returns 200 HTTP status code with response body as `PowerConsumptionResponse` schema. For more information, see [VPP API Reference](vpp_api_reference.html). #### Example Response ```json { "data" : { "PowerConsumption": [1,2,4], "StartTime": "2020-02-05T23:15:00Z", "TimeSlotLength": 20 } } ``` # Virtual Power Plant API # Digital Logistics documentation # Contact For general inquries, including obtaining an API key, please contact the Siemens Digital Logistics team at: Email: [plm.sdl.disw@siemens.com](mailto:plm.sdl.disw@siemens.com) # FAQ ***Why do I need to email someone for API access?*** We are actively exploring more efficient methods for granting access to the Carbon Footprint Indicator for Transportation API. The email-based process is a temporary measure to ensure access while we develop a superior solution. ***GLEC is the current standard, but there will be a new standard in the EU based on ISO 14083 / ISAE3000. Therefore, it will be important to meet these standards as well. Reason is that in the EU, companies will need to pay tax on CO2 starting in 2025 or so (the timeline is different per country). The tools to calculate CO2 emissions will need to be certified for this to occur. Is that correct?*** Nope! New ISO Standard: ISO 14083 “Quantification and reporting of greenhouse gas emissions of transport operations.” The intention is that the principles and methodology for freight transport will be based on, and consistent with, the GLEC Framework. Once in place the new ISO Standard is expected to align with Smart Freight Centre's in conformance with the GLEC Framework’ accreditation of programs and tools. This means that companies currently using accredited programs and tools won’t need to make changes in 2022. See also: [iso.org documentation on ISO 14083](https://www.iso.org/standard/78864.html) # Getting Started To begin using the **Transport Carbon Calculator**, follow these steps: 1. Contact our support team to obtain an API Key. You can find our contact information [contact](contact.html). 1. Include your API Key in the `x-api-key` header of your HTTP requests. Note See the [API Specifications](technical-docs.html) for more information. ## Example Request ```bash API_KEY = your-api-key # Calculates CO2e emission for a single leg. curl -X 'POST' \ 'https://siemens.supplychainsuitedev.net/publicapi/master/demo/tcc/1.0.0/calculate' \ -H 'accept: application/json' \ -H 'x-api-key: API_KEY' \ -H 'Content-Type: application/json' \ -d '{ "legs": [ { "shipmentId": "2025", "legId": "1", "senderLocationId": "2025_PL_41300", "recipientLocationId": "2025_DE_34127", "transportMode": "road", "shipmentDate": "2025-11-11T00:00:00Z", "distanceKm": null, "weightKg": 25000, "volumeCbm": null, "loadingMeterM": null, "equipment": null, "fuelType": null, "fuelConsumption": null, "fuelConsumptionUnit": null } ], "parameters": [ { "key": "bEnableTestMode", "value": "1" } ], "locations": [ { "id": "2025_PL_41300", "country": "PL", "postalCode": "41300", "city": null, "street": null, "houseNumber": null, "latitude": null, "longitude": null }, { "id": "2025_DE_34127", "country": "DE", "postalCode": "34127", "city": null, "street": null, "houseNumber": null, "latitude": null, "longitude": null } ] }' ``` # User Guide ## Transport Carbon Calculator Restricted | © Siemens 2025 ______________________________________________________________________ ## Contents - [About Transport Carbon Calculator (TCC)](#about-transport-carbon-calculator-tcc) - [Input and Output Contracts](#input-and-output-contracts) - [Input](#input) - [Output](#output) ______________________________________________________________________ ## About Transport Carbon Calculator (TCC) The Transport Carbon Calculator (TCC) web service calculates transport emissions for single-leg relations between two locations, in conformance with the Global Logistics Emissions Council (GLEC) framework. TCC enables worldwide carbon equivalent emission calculations with minimal input requirements. It provides two methodology options: - **Direct methodology** - based on actual fuel consumption. - **Default methodology** - based on emission factors. TCC supports all commonly used transport channels: rail, road, sea, and air. Additionally, it calculates emissions produced at logistics sites, such as during transshipment processes. Users can select various equipment types for each transport channel, allowing for a more precise description of the transport and, consequently, a more accurate emission calculation. The option to choose between different fuel types further enhances the quality and accuracy of the calculation process. Transport Carbon Calculator offers: - Leg-based calculation of carbon equivalent emissions - Fast and easy integration of GLEC methodology - Direct or default calculation options - Support for various equipment inputs (fuel, vehicle, etc.) - Coverage of multiple transport channels (rail, road, sea, air, logistics sites) - Worldwide coverage - Integrated geocoding and georouting capabilities ### GLEC Framework The CO2 equivalents (CO2e) calculation module used by the service is accredited by Smart Freight Centre for being conform with the GLEC methodology as well as ISO 14083, ensuring the accuracy and reliability of the calculations. For more information about the GLEC Framework see [Smart Freight Center](https://www.smartfreightcentre.org/en/our-programs/emissions-accounting/global-logistics-emissions-council/calculate-report-glec-framework/). ______________________________________________________________________ ## Input and Output Contracts ### Default values for equipment and fuel type | transportMode | validFrom | validTo | equipment | fuelType | | ------------- | ---------- | ---------- | ------------------ | -------- | | road | 01.01.1900 | 31.12.9999 | Large Truck | Diesel | | air | 01.01.1900 | 31.12.9999 | freighter | Kerosene | | rail | 01.01.1900 | 31.12.9999 | | | | logisticsite | 01.01.1900 | 31.12.9999 | transhipment_mixed | | | sea | 01.01.1900 | 31.12.9999 | 20ft | HFO | ______________________________________________________________________ ## Input ### Table "legs" | Name | Type | Values | Mandatory | Explanation | | ------------------- | ----- | ------------------------------------------------------ | --------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | shipmentId | str | | yes | The identifier for the shipment. Case sensitive | | legId | str | | yes | The identifier for the transport legs. Case sensitive | | senderLocationId | str | | yes | Id of the sender location. Link to locations table | | recipientLocationId | str | | yes | Id of the recipient location. Link to locations table | | transportMode | str | ["Road", "Sea", "Air", "Rail", "logisticsite"] | yes | Used transport mode. Choices: "logisticsite", "road", "air", "sea", "rail" | | shipmentDate | date | Any date in format "yyyy-mm-dd". Example: "2030-12-24" | no | Shipment date. If not date is given, current date will be used | | distanceKm | float | | no | Distance of the transport leg. Must be given in km. If not given, will be calculated | | weightKg | float | | yes | Gross transport weight including packaging. Must be given in kg | | volumeCbm | float | | no | Gross volume including packaging. Must be given in cbm | | loadingMeterM | float | | no | Gross loading meter including packaging. Must be given in meter | | equipment | str | choices see below | no | Equipment. Choices depend on fuelType and transportMode | | fuelType | str | choices see below | no | Fuel used. Choices depend on equipment and transportMode | | fuelConsumption | float | | no | Consumption. If given, is used to calculate the CO2e. If given fuelType must be given. Unit of measurement depends on chosen fuelType | | fuelConsumptionUnit | str | choices see below | no | Unit of the fuelConsumption. Choices: "kwh/100km" (for electric), "l/100km" (for diesel, lpg, b5-diesel, kerosene), "kg/100km" (for diesel, lpg, b5-diesel, lng, cng, kerosene, mdo, hfo) | ## Allowed Values for Different Columns in the Table "Legs" ### Road #### Equipment for Transport Mode Road | Valid From | Valid To | Equipment | Fuel Type | | ---------- | ---------- | ------------ | --------- | | 01.01.1900 | 31.12.9999 | Sprinter | Diesel | | 01.01.1900 | 31.12.9999 | Small Truck | Diesel | | 01.01.1900 | 31.12.9999 | Medium Truck | Diesel | | 01.01.1900 | 31.12.9999 | Large Truck | Diesel | | 01.01.2025 | 31.12.9999 | Sprinter | Electric | | 01.01.2025 | 31.12.9999 | Small Truck | Electric | | 01.01.2025 | 31.12.9999 | Medium Truck | Electric | | 01.01.2025 | 31.12.9999 | Large Truck | Electric | | 01.01.2025 | 31.12.9999 | Sprinter | CNG | | 01.01.2025 | 31.12.9999 | Small Truck | CNG | | 01.01.2025 | 31.12.9999 | Medium Truck | CNG | | 01.01.2025 | 31.12.9999 | Large Truck | CNG | | 01.01.2025 | 31.12.9999 | Large Truck | LNG | #### Fuel Consumption Unit for Transport Mode Road - Kwh / 100 km (for fuelType Electric) - L / 100 km (for fuelType Diesel, LPG, B5-diesel) - Kg / 100 km (for fuelType Diesel, LPG, B-diesel, LNG, CNG) #### Fuel Type for Transport Mode Road - Diesel - LPG - B5-diesel - Electric - LNG - CNG ### Rail #### Equipment for Transport Mode Rail | Valid From | Valid To | Equipment | Fuel Type | | ---------- | ---------- | ---------------------- | --------- | | 01.01.1900 | 31.12.9999 | | diesel | | 01.01.1900 | 31.12.9999 | `light` | diesel | | 01.01.1900 | 31.12.9999 | `average` | diesel | | 01.01.1900 | 31.12.9999 | `extralarge` | diesel | | 01.01.1900 | 31.12.9999 | `heavy` | diesel | | 01.01.1900 | 31.12.9999 | `car` | diesel | | 01.01.1900 | 31.12.9999 | `chemistry` | diesel | | 01.01.1900 | 31.12.9999 | `container` | diesel | | 01.01.1900 | 31.12.9999 | `coalandsteel` | diesel | | 01.01.1900 | 31.12.9999 | `buildingmaterials` | diesel | | 01.01.1900 | 31.12.9999 | `manufacturedproducts` | diesel | | 01.01.1900 | 31.12.9999 | `cereals` | diesel | | 01.01.1900 | 31.12.9999 | `highspeed` | diesel | | 01.01.1900 | 31.12.9999 | `doublestackcontainer` | diesel | | 01.01.1900 | 31.12.9999 | `rollingroad` | diesel | | 01.01.1900 | 31.12.9999 | | electric | | 01.01.1900 | 31.12.9999 | `light` | electric | | 01.01.1900 | 31.12.9999 | `average` | electric | | 01.01.1900 | 31.12.9999 | `extralarge` | electric | | 01.01.1900 | 31.12.9999 | `heavy` | electric | | 01.01.1900 | 31.12.9999 | `average` | electric | | 01.01.1900 | 31.12.9999 | `extralarge` | electric | | 01.01.1900 | 31.12.9999 | `heavy` | electric | | 01.01.1900 | 31.12.9999 | `car` | electric | | 01.01.1900 | 31.12.9999 | `chemistry` | electric | | 01.01.1900 | 31.12.9999 | `container` | electric | | 01.01.1900 | 31.12.9999 | `coalandsteel` | electric | | 01.01.1900 | 31.12.9999 | `buildingmaterials` | electric | | 01.01.1900 | 31.12.9999 | `manufacturedproducts` | electric | | 01.01.1900 | 31.12.9999 | `cereals` | electric | | 01.01.1900 | 31.12.9999 | `highspeed` | electric | | 01.01.1900 | 31.12.9999 | `doublestackcontainer` | electric | | 01.01.1900 | 31.12.9999 | `rollingroad` | electric | #### Fuel Consumption Unit for Transport Mode Rail - NA #### Fuel Type for Transport Mode Rail - NA ### Air #### Equipment for Transport Mode Air | Valid From | Valid To | Equipment | | ---------- | ---------- | ------------- | | 01.01.1900 | 31.12.9999 | Unknown | | 01.01.1900 | 31.12.9999 | Belly Freight | | 01.01.1900 | 31.12.9999 | Freighter | #### Fuel Consumption Unit for Transport Mode Air - L / 100 km - Kg / 100 km #### Fuel Type for Transport Mode Air - Kerosene ### Sea #### Equipment for Transport Mode Sea | Valid From | Valid To | Equipment | | ---------- | ---------- | ------------- | | 01.01.1900 | 31.12.9999 | 20ft_Reefer | | 01.01.1900 | 31.12.9999 | 20ft | | 01.01.1900 | 31.12.9999 | 40ft_Reefer | | 01.01.1900 | 31.12.9999 | 40ft | | 01.01.1900 | 31.12.9999 | 40ftHC_Reefer | | 01.01.1900 | 31.12.9999 | 40ftHC | #### Fuel Consumption Unit for Transport Mode Sea - Kg / 100 km #### Fuel Type for Transport Mode Sea - MDO - HFO ### Logisticsite #### Equipment for Transport Mode Logisticsite | Valid From | Valid To | Equipment | | ---------- | ---------- | ---------------------------- | | 01.01.1900 | 31.12.9999 | transhipment_ambient | | 01.01.1900 | 31.12.9999 | transhipment_mixed | | 01.01.1900 | 31.12.9999 | storage_transhipment_ambient | | 01.01.1900 | 31.12.9999 | storage_transhipment_mixed | | 01.01.1900 | 31.12.9999 | warehouse_ambient | | 01.01.1900 | 31.12.9999 | warehouse_mixed | | 01.01.1900 | 31.12.9999 | container_terminal_ambient | | 01.01.1900 | 31.12.9999 | container_terminal_mixed | | 01.01.1900 | 31.12.9999 | liquid_bulk_terminal_ambient | | 01.01.1900 | 31.12.9999 | liquid_bulk_terminal_mixed | #### Fuel Consumption Unit for Transport Mode Logisticsite - NA #### Fuel Type for Transport Mode Logisticsite - NA ## Table "Locations" | Name | Type | Mandatory | Explanation | | ----------- | ----- | --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | id | str | yes | The unique identifier for the location. | | country | str | no/yes | The country of the location in ISO 3166-1 alpha-2 format, e.g. “DE” for Germany. Either country+postalCode+city is given or latitude+longitude. Parsing case insensitive. | | postalCode | str | no/yes | The postcode of the location. Either country+postalCode+city is given or latitude+longitude. | | city | str | no/yes | The city name of the location. Either country+postalCode+city is given or latitude+longitude. | | street | str | no | The street name of the location. | | housenumber | str | no | The housenumber of the location. | | latitude | float | no/yes | The latitude of the location. In decimal format. Either country+postalCode+city is given or latitude+longitude. | | longitude | float | no/yes | The longitude of the location. In decimal format. Either country+postalCode+city is given or latitude+longitude. | ## Table "Parameters" | Name | Type | Mandatory | Explanation | | ----- | ---- | --------- | ----------------------- | | key | str | no | Name of the parameter. | | value | str | no | Value of the parameter. | ## Output ### Table "resultsLegs" | Key | Value | Description | | ------------- | ----- | -------------------------------------------------------- | | shipmentId | str | The identifier for the shipment. | | legId | str | The identifier for the transport legs. | | transportMode | str | Transport mode used for this leg. | | co2eKg | float | Calculated Well-to-Wheel CO2e emissions in kg. | | wttCo2eKg | float | Calculated Well-to-Tank CO2e emissions in kg. | | ttwCo2eKg | float | Calculated Tank-to-Wheel CO2e emissions in kg. | | errorSeverity | str | Highest severity error type (Error, Warning, Hint, None) | ### Table "resultLocations" | Key | Value | Description | | ------------- | ----- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- | | id | str | The unique identifier for the location. Is created from shipment id, country code, and postal code. Ids for the transshipment locations are IATA or UN/LOCODES. | | latitude | float | The latitude of the location. | | longitude | float | The longitude of the location. | | errorSeverity | str | Highest severity error type (Error, Warning, Hint, None) | ### Table "resultMessages" | Key | Value | Description | | -------- | ----- | ------------------------------------------------------------------------------ | | table | str | Name of the table to which the entry refers. | | causer | str | Depending on the Table, this field contains the shipment, leg, or location id. | | message | str | Message that explains the issue. | | severity | str | Severity of the error (Error, Warning, or Hint) | | code | str | Error code - see below | If output errors occur in the result, a detailed message is given. There are three different error types: - **Hint**: Hint for the user, which might be interesting, but with no effect on the results. - **Warning**: Result could be calculated, but a workaround has been used. - **Error**: Result could not be calculated. The tables resultShipments, resultLegs, resultLocations only show the type, while the respective error message is listed in the resultErrors table. A data row can produce multiple errors. The affected table only shows the most severe error type, while all error types and messages are listed in the resultErrors table. #### Output Errors for Table "resultLegs" | Type | Code | Message | | ------- | ------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Error | resultLegs_noShipmentId | "No shipmentId is given. Leg will not be processed further." | | Error | resultLegs_noLegId | "No legId is given. Leg will not be processed further." | | Error | resultLegs_legIdNotUnique | "The leg " + legId + " of the shipment (" + shipmentId + ") is not unique. Leg will not be processed further." | | Error | resultLegs_noSenderId | "For the leg " + legId + " of the shipment (" + shipmentId + ") no senderId is given. Leg will not be processed further." | | Error | resultLegs_noRecipientId | "For the leg " + legId + " of the shipment (" + shipmentId + ") no recipientId is given. Leg will not be processed further." | | Error | resultLegs_noDate | "For the leg " + legId + " of the shipment (" + shipmentId + ") no date is given. Leg will not be further proceeded." | | Error | resultLegs_dateTooLate | "For the leg " + legId + " of the shipment (" + shipmentId + ") the date is more than 15 years in the future. This is not covered. Leg will not be further proceeded." | | Warning | resultLegs_dateTooEarly | "For the leg " + legId + " of the shipment (" + shipmentId + ") the date is before the 01.01.2022. This is not covered. The 01.01.2022 will be used as leg data." | | Error | resultLegs_noWeightKg | "For the leg " + legId + " of the shipment (" + shipmentId + ") either no weightKg is given or weightKg \<= 0 kg. Leg will not be processed further." | | Hint | resultLegs_weightKgTooHeavy | "For the leg " + legId + " of the shipment (" + shipmentId + ") the weightKg > 25000 kg." | | Hint | resultLegs_weightKgTooLight | "For the leg " + legId + " of the shipment (" + shipmentId + ") the weightKg < 1 kg." | | Hint | resultLegs_noDistanceKm | "For the leg " + legId + " of the shipment (" + shipmentId + ") no distanceKm is given. Distance will be calculated." | | Hint | resultLegs_distanceKm\<=0 | "For the leg " + legId + " of the shipment (" + shipmentId + ") the distanceKm \<= 0 km. Distance will be recalculated." | | Error | resultLegs_senderIdGeoInformationMissing | "For the leg " + legId + " of the shipment (" + shipmentId + ") the sender geocoding information is missing. The leg will not be further processed." | | Error | resultLegs_recipientIdGeoInformationMissing | "For the leg " + legId + " of the shipment (" + shipmentId + ") the recipient geocoding information is missing. The leg will not be further processed." | | Error | resultLegs_noKnownTransportMode | "For the leg " + legId + " of the shipment (" + shipmentId + ") no known transport mode is given (choices are 'air', 'sea', 'road', 'inland water', 'rail', 'logisticsite'). The leg will not be further processed." | | Error | resultLegs_noFuelTypeButConsumption | "For the leg " + legId + " of the shipment (" + shipmentId + ") a consumption has been defined, but not a fuelType. The leg will not be further processed." | | Warning | resultLegs_noKnownFuelType | "For the leg " + legId + " of the shipment (" + shipmentId + ") no known fuel type has been selected. The default will be used. Choices are: "+Choices_FuelType_noConsumption if fuelType not in ["", None] and fuelType not in Choices_FuelType_noConsumption and not fuelConsumption else "For the leg " + legId + " of the shipment (" + shipmentId + ") no known fuel type has been selected. The default will be used. Choices are: "+Choices_FuelType_Consumption if fuelType not in Choices_FuelType_Consumption and fuelConsumption else | | Warning | resultLegs_wrongEquipmentRoad | "For the leg " + legId + " of the shipment (" + shipmentId + ") the equipment ("+equipment+") is not valid for transportMode (" + transportMode +"). Choices road: "+Choices_Road | | Warning | resultLegs_wrongEquipmentRail | "For the leg " + legId + " of the shipment (" + shipmentId + ") the equipment ("+equipment+") is not valid for transportMode (" + transportMode +"). Choices rail: "+Choices_Rail | | Warning | resultLegs_wrongEquipmentInlandWater | "For the leg " + legId + " of the shipment (" + shipmentId + ") the equipment ("+equipment+") is not valid for transportMode (" + transportMode +"). Choices inlandwaterways: "+Choices_InlandWaterways | | Warning | resultLegs_wrongEquipmentSea | "For the leg " + legId + " of the shipment (" + shipmentId + ") the equipment ("+equipment+") is not valid for transportMode (" + transportMode +"). Choices sea: "+Choices_Sea | | Warning | resultLegs_wrongEquipmentAir | "For the leg " + legId + " of the shipment (" + shipmentId + ") the equipment ("+equipment+") is not valid for transportMode (" + transportMode +"). Choices are: "+Choices_Air | | Warning | resultLegs_wrongEquipmentLogisticSite | "For the leg " + legId + " of the shipment (" + shipmentId + ") the equipment ("+equipment+") is not valid for transportMode (" + transportMode +"). Choices are: "+Choices_LogisticSites | | Warning | resultsLegs_noKnownFuelConsumptionUnit | "For the leg " + legId + " of the shipment (" + shipmentId + ") a unknown fuelConsumptionUnit has been given, while a fuelConsumption is given. The default unit will be used. Choices are: "+Choices_FuelConsumptionUnit | | Warning | resultLegs_noConsumptionButUnit | "For the leg " + legId + " of the shipment (" + shipmentId + ") a fuelConsumptionUnit has been given, while a fuelConsumption is not given. The unit will be ignored." | | Hint | resultLegs_noDirectConsumptionForRail | "For the leg " + legId + " of the shipment (" + shipmentId + ") a consumption has been defined, but the transport mode 'rail' is not supported for direct emission calculation. The default emission calculation will be used." | | Error | resultLegs_senderIdRecipientIdNoRegionFound | "For the leg "+ legId + " of the shipment (" + shipmentId + ") the sender country "+Sender_country+" and the recipient country "+Recipient_country+" could not be assigned to any region. Please check the country codes." | | Error | resultLegs_senderIdNoRegionFound | "For the leg "+ legId + " of the shipment (" + shipmentId + ") the sender country "+Sender_country+ " could not be assigned to any region. Please check the country code." | | Error | resultLegs_senderIdNoRegionFound | "For the leg "+legId+ " of the shipment (" + shipmentId + ") the recipient country "+Recipient_country+" could not be assigned to any region. Please check the country code." | | Error | resultLegs_senderGeocodingFailed | "For the leg "+legId+" of the shipment (" + shipmentId + ") the sender could not be geocoded. The leg will not be further processed." | | Error | resultLegs_recipientGeocodingFailed | "For the leg "+legId+" of the shipment (" + shipmentId + ") the recipient could not be geocoded. The leg will not be further processed." | | Error | resultLegs_senderRecipientGeocodingFailed | "For the leg "+legId+" of the shipment (" + shipmentId + ") the sender and recipient could not be geocoded. The leg will not be further processed." | | Error | resultLegs_noDistanceCalculated | "For leg "+legId+" of the shipment (" + shipmentId + ") no distance could be calculated." | | Error | resultLegs_co2eNotCalculated | "For leg "+legId+" of the shipment (" + shipmentId + ") the CO2e emission calculation has failed." | | Warning | resultLegs_riskIndexNotCalculated | "The risk index for the leg "+legId+" of the shipment (" + shipmentId + ") could not be calculated." | | Error | resultLegs_routingFailed | "No routing was possible because origin and destination location are within different sub-networks: [...]" "[...] Could not find transferpoints [...]" | | Error | resultLegs_costsNotCalculated | "Transport costs for the leg " + legId + " of the shipment (" + shipmentId + ") could not be calculated. Please check the input." | #### Output Errors for Table "resultLocations" | Type | Code | Message | | ------- | ------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Hint | resultLocations_locationNotUsed | "Location is not used in the shipments table. Therefore, it is not further processed." | | Error | resultLocations_geocodingNotPossible | "For the location " + id + " address information is missing or false. Legs containing this location will not be further processed." | | Warning | resultLocations_notUniqueId | "The location " + id + " appears more than once with different addresses in the data. Please correct. Different addresses should have different ids. In the further calculation a random address will be selected." | | Error | resultLocations_geocodingFailed | "The location "+id+" could not be geocoded." | | Error | resultLocations_regionNotFound | "For the location "+id+" the country "+country+" could not be assigned to any region. Please check the country code." | - [Transport Carbon Calculator](home.html) - [Getting Started](getting-started.html) - [Rest API](technical-docs.html) - [FAQ](faq.html) - [Contact](contact.html) # OpenAPI Specifications [Download OpenAPI Specification](spec.json) # Drivetrain Analyzer documentation # Drivetrain Analyzer API Drivetrain Analyzer Cloud allows you to programmatically read the data of your connected motor fleet via [Insights Hub Services and Industrial loT APIs](../insights-hub/overview.html). ## Getting technical token for accessing Insights Hub APIs You require the Drivetrain Analyzer Cloud API key for your tenant. You can find this in the Settings page. Here, you can view your API key and the expiration date of your API key. The system sends you an email if the expiration date is to expire within 0 to 7 days or has already expired. You can generate your new API key if it is to expire within 0 to 7 days. If the expiration date is longer than 7 days, then button "Generate" is deactivated. If an API key is not displayed in the dialog after you click on "Generate", then you are notified that you should contact user support. > ***NOTE:*** > > **Security measures** > > Drivetrain Analyzer Cloud API key must be kept secret. > > - Ensure that the transport of the key is secured by technical measures such as encrypted/signed emails, encrypted/signed USB flash drives, etc., specially >in public areas such as the Internet. > - Store the key in the OEM/end customer area in such a way that it is protected from unauthorized access (e.g. on SharePoints, databases, etc. through user >management with access data). With the Drivetrain Analyzer Cloud key, request your bearer token (access token) using the following **HTTP request:** ```text GET /public/iqtoken/token?key={your Drivetrain Analyzer Cloud API key} HTTP/1 .1 ``` **Host:** Region - Europe 1 (Insights Hub on AWS): iqtoken-visualflowcreatorhttp.eul.mindsphere.io **HTTP Response:** ```json { "access_token": {Bearer Token}, "token_type": "bearer", "expires_in": 1799, "scope": "mdsp:core:Admin3rdPartyTechUser", "jti": "9d53d67bb046437d90c309c3a228d942", "timestamp": 1613126706928 } ``` Use the {bearer token} to call the Industrial loT APIs and access your client tenant's data. > ***More Information:*** > > More information on calling up the Industrial loT Endpoints is provided in the Insights Hub documentation (). ## Reading assets and asset properties for the EU1 region (Insights Hub running on AWS) The following provides an overview of the data that you can access as a user of your client tenant. ### Drivetrain Analyzer Cloud assets #### Asset variables Drivetrain Analyzer Cloud assets are of the type "Simotics" (core.basicdevice -> tenant_name."Simotics"). - AgentSerialNumer: serial number of the Connection Module IOT device - Application: motor application Possible values are: Pump, fan, compressor, other - AssetTypeVersion: asset type version number - BatteryRuntimeLeft: estimate of the remaining battery time in months - BatteryStatus: battery status as a percentage (%) - BearingCondition: Motor bearing condition Possible values are: {0 = calibration/missing data, 1 = OK , 2 = low/medium probability of failure, 3 = high probability of failure} - Conditionstate: asset status that is derived from the open reports in the logbook. Possible values are: {0 = OK, 1 = info, 2 = warning, 3 = error, 4 = unknown} - Connected At: date and time of the last data that Connection Module IOT transferred to SI IQ Fleet - Connectionstate: Connection Module IOT connection status. Possible values: {0 = connected, 1 = temporarily interrupted, 2 = disconnected} - CreateDate: date and time that the asset was created - DigitalTwinVersion: (internal) motor digital twin version, which is used during the onboarding - ElectricalDatalndex: selected motor operating point index - FirmwareVersion: installed firmware version of Connection Module IOT - GenericVibrationMonitoring: true, if Connection Module IOT was onboarded as sensor for generic vibration monitoring. - ManufacturerProductNumber: manufacturer's product article number (MLFB) of the motor - OperatingMode: Motor operating mode Possible values are: {1 = DOL, 2 = VSD} - PoleNumber: number of motor poles - Relubricationinterval: motor relubrication interval in hours - ScanCycle: Connection Module IOT sensor measurement interval in the "duration" format defined in standard ISO 8601 - SerialNumber: motor serial number - TransmitCycle: Connection Module IOT data transfer interval in the "duration" format defined in standard ISO 8601 - VoltageDropResets: number of Connection Module IOT brownout resets You can call up the list of assets integrated in the application and the variables of each asset using the "Asset Management Service" of Insights Hub. The documentation and the API specification can be found at the following link () **Example:** ```text GET https://gateway.eu1.mindsphere.io/api/assetmanagement/v3/assets ``` Header: ```text Content-Type: application/json Authorization: Bearer ``` ### Signals All Time Series data of the IQFIeet signaIs are available in the aspects of the "Simotics" asset type: In order to read out the SIDRIVE IQ Fleet signals, all data points within a time period are provided by the "IOT Time Series Service" of Insights Hub. The description and the API specification can be found at the following link: () As an alternative to individual data points, you can also use the "Aggregates Service" of Insights Hub. The description is provided at the following link: () **Example - IOT Time Series Service:** ```text GET https://gateway.eu1.mindsphere.io/api/iottimeseries/v3/timeseries/789ffa1f911f4cc08e5a8021208d9476/LowFrequency?to=2020-07-11T07:00:00.00Z&sort=desc ``` Header: ```text Content-Type: application/json Authorization: Bearer ``` **Example - Aggregates Service:** ```text GET https://gateway.eu1.mindsphere.io/api/iottsaggregates/v3/aggregates/daa80a036ad74578b49c72e76ded3a24/LowFrequency?select=ElectricalEnergy&intervalUnit=hour&from=2020-05-01T00:00:00.000Z&to=2020-06-23T08:00:00.00Z&intervalValue=8 ``` Header: ```text Content-Type: application/json Authorization: Bearer ``` ### Notifications #### Logs **Event type:** IQFIeetEvent\ **Event Type Id:** 015578eb-c77a-4f85-8ac7-8fbbd6e1d181 You can access all protocols that were created for the assets of the tenant via the "Event Management Service" of Insights Hub. The documentation and the API specification can be found at the following link: () **Example:** ```text GET https://gateway.eu1.mindsphere.io/api/eventmanagement/v3/events?filter={%22and%22:{%22timestamp%22:{%22after%22:%222019-04-26T21:00:00.000Z%22},%22typeId%22:%22015578eb-c77a-4f85-8ac7-8fbbd6e1d181%22}} ``` Header: ```text Content-Type: application/json Authorization: Bearer ``` **Example response model:** ```json { "id": "7d7f518f-5962-40e6-bbdd-6d82a48f977d", "typeld": "015578eb-c77a-4f85-8ac7-8fbbd6eldl81", "correlationld": "63884d7db29503c577262efa351192b3", "timestamp": "2022-12-01T00:45:45Z", "entityld": "b3cf3b0191d64fb5a5clebedl91d2506", "etag": 0, "_links": { "self": { "href": "https://gateway.eul.mindsphere.io/api/eventmanagement/v3/events/7d7f518f-5962-40e6-bbdd-6d82a48f977d" } }, "logType": "multiplethresholdviolation", "severity": 20, "kpiValue": "14.0674", "acknowledged": false, "kpiName": "RMSVelY", "description": "Error Threshold Connect Vibration Tangential(y)", "thresholdvalue": "10", "date2": "2022-12-01T06:44:39.000Z", "datel": "2022-12-01T00:45:45.000Z", "title": "Error Threshold Connect Vibration Tangential (y)", "serviceTag": false, "comment": "", "detail": "Between 12/1/2022 12:45:45 AM and 12/1/2022 6:44:39 AM, the Connect Vibration Tangential (y) error threshold value 10 mm/s has been exceeded by 46 data point(s) with the maximum value of 14.067 mm/s.", "category": "", "parameterl": "46" } ``` #### Comments **Event Type:** IQFIeetCommentEvent\ **EventTypeld:** 953b41 43-fc09-4ebe-a05d-eeb0a5c6af34 **Example response model:** ```json { "id": "3cc0f416-7f7a-41a8-b005-390c6d9def0d", "typeld": "953b4143-fcO 9-4ebe-a05d-eeb0a5c6af34", "correlationld": "638864a59080059201c35254c34fdc21", "timestamp": "2022-12-01T08:24:05.114Z", "entityld": "b3cf3b0191d64fb5a5clebedl91d2506", "etag": 0, "_links": { "self": { "href": "https://gateway.eul.mindsphere.io/api/eventmanagement/v3/events/3cc0f416-7f7a-41a8-b005-390c6d9def0d" } }, "severity": 0, "parentLogld": "7d7f518f-5962-40e6-bbdd-6d82a48f977d", "acknowledged": false, "serviceTag": false, "userEmail": "user@email.com", "commentText": "This is a comment" } ``` # Contact Please use this email address for questions or requests related to Drivetrain Analyzer Cloud, SIMOTICS CONNECT 400 or Connection Module IOT: Email: [support.digital.drives@siemens.com](mailto:support.digital.drives@siemens.com) - [Overview](overview.html) - [Getting started](start.html) - [API Reference](api.html) - [Contact](contact.html) # Overview The drivetrain is core in countless manufacturing processes. Malfunctions or failures of motors, drives, gearboxes etc. therefore result in costly downtimes. Whether frequency converters, motors, or other rotating and non-rotating components of your machine – Drivetrain Analyzer Cloud allows you to monitor the condition of your equipment, analyze and optimize your drivetrain health and energy consumption, enabling easy predictive maintenance. **Leverage the cloud for improving your drivetrain operations:** Drivetrain Analyzer Cloud allows you to easily avoid unplanned downtimes and reduce costs, all with the help of the cloud. - **Fleet Management:** Monitor the condition of your distributed assets from anywhere in the world. 24/7 available. - **Optimize Health:** Advanced AI allows to analyze the health of your motors and equipment. See the motor and pump health in the form of an easy-to-understand traffic light, allowing for predictive maintenance. - **Improve Energy:** Observe CO2 emissions, energy usage and costs related to your assets. You even get recommendations for improvements, helping you to save money and energy. ## Drivetrain Analyzer Cloud The SIMOTICS CONNECT 400 or CONNECTION MODULE IOT sensor module connects drivetrain components with the cloud-based application Drivetrain Analyzer. This enables you to improve the productivity, reliability, and service options of your drivetrain. ​You can transfer operating data to Drivetrain Analyzer Cloud where it is saved and analyzed via a CONNECTION MODULE IOT or SIMOTICS CONNECT 400 sensor module. Drivetrain Analyzer shows the user the operating data of the asset and the results of the operation and status analysis and provides, for example, recommendations for predictive maintenance activities. In addition to the SIMOTICS CONNECT 400 and Connection Module IOT, Drivetrain Analyzer Cloud also offers the possibility to connect and monitor SINAMICS drives in the cloud. This enables holistic drivetrain monitoring, from mechanical to electrical, all in the same dashboard and application. In addition, the CMS1200 Condition Monitoring System can be connected to the cloud, enabling continuous and accurate monitoring of the condition of mechanical components in your drivetrain using wired sensors. ## Motor Monitoring In the fleet application area, you can monitor Siemens and 3rd party low-voltage motors that are connected to Drivetrain Analyzer Cloud via the SIMOTICS CONNECT 400 or Connection Module IOT sensor module. The data that you read out with Drivetrain Analyzer Cloud allows you to view several assets in an overview. Among other things, you can monitor and read out the following information in the application area: - Status information - Asset information - Various signals, for example: - Speed - Electrical power - Information related to preventive maintenance - Alarms and warnings The Fleet application area offers you various functions for identifying and evaluating an asset: - Localization of the asset using a map - Asset filter function - Visualization of signals over a period of time - Logbook function (to document maintenance measures) - Profile of the motor profile - Central access to product-specific support pages (SIOS, Spares on Web, etc.) - Setting the email notification and display of the Connection Module IOT device data - Thresholds for the output of warnings and errors - Exporting of asset data - Supporting maintenance planning with operating hour-based time schedules ## Generic Vibration Monitoring In addition to low-voltage motors, you can mount Connection Module IOT on other components of the drive train. These include the following components: - Gearbox - Coupling - Pillow block bearing - Pump - Fan - Compressor - Other rotating and non-rotating equipment For these components, in Drivetrain Analyzer Cloud you can configure the assets for "Generic vibration monitoring". The monitoring is based on temperature and vibration signals. > ***NOTE:*** The function is limited to temperature and vibration monitoring. Currently, only restricted analysis functions are available for "Generic vibration monitoring". The Fleet application area is adapted to the functional scope of "Generic vibration monitoring". ## Drive Monitoring In the Fleet application area, you can monitor drives that are connected to Drivetrain Analyzer Cloud via the Insights Hub connectivity solution. The data that you read out via the Fleet application area permits you to view several assets in an overview. Among other things, you can monitor and read out the following information in the Fleet application area: - Status information - Asset information - Various signals, for example: - Active power - Speed - Information related to preventive maintenance - Alarms and warnings # Getting Started With the Drivetrain Analyzer Cloud app and the plug-and-play Connection Module IOT, you always have a detailed overview of the current health state of your system from different perspectives. This allows you to detect anomalies before they cause problems. You receive timely warnings about potential component failures to avoid unplanned downtime and plan repairs. Please find all relevant information [on siemens.com](https://siemens.com/dta-cloud). # Edge App SDK documentation For any Product and business questions please contact: [foa.tfoasasin.in@siemens.com](mailto:foa.tfoasasin.in@siemens.com) - [Overview](overview.html) - [Contact](contact.html) ## Welcome to the Edge App Software Development Kit (Edge App SDK) Product page The Edge App SDK speeds up and enhances the entire development lifecycle of Edge apps from creation to deployment. This allows the developer to focus more on the business logic and offload the app creation, generation, and deployment to SDK. You can develop apps on your development machine/VM without the need for a physical Industrial Edge device or Industrial Edge Management. Additionally, the SDK provides a data simulator to generate mock data for app testing and debugging. ## Download and Install the Edge App SDK To get started with the Edge App SDK, download and install it from the following link: [Download Edge App SDK](https://support.industry.siemens.com/cs/ww/en/view/109976101) ## WHO Are Our End Customers? 1. **Users Who Buy and Use Siemens/Partner Apps**: These users rely on marketplace applications to enhance their industrial operations. 1. **Users Who Build Custom Apps**: App Partners and OEMs who create bespoke solutions tailored to specific industrial needs. ## WHAT Are the Problems/Gaps Our Customers Face? - **Scattered Edge App Development Processes**: Developing edge applications for industrial devices has been fragmented and inefficient. - **No Capabilities Available for In-Line Testing**: Developers struggle with the iterative process of deploying, testing, debugging, and redeploying apps. - **No Resource Allocation/Management Suggestions at the Development Stage**: Inefficient resource management during development leads to potential issues later in the process. - **Huge Learning Curve**: The complexity of existing tools and processes increases the learning curve, making it difficult for developers to get started. ## HOW Can We Address the Gaps and Help Our Customers? - **Speeding Up App Development**: The Edge App SDK accelerates development through App Skeleton Code Generation, allowing developers to focus more on business logic. - **Saving Time in Setting Up Artifacts**: Automated app artifact generation reduces manual effort, making the process more efficient. - **Mitigating the Overhead of Configurations**: The SDK simplifies configurations, creating a more streamlined experience for developers. - **Automating .app Builds**: Automation ensures consistency and reliability during the packaging phase of application development. - **Optimizing Resource Utilization**: The SDK provides tools for resource management, optimizing computing resource use during development and execution. ## Edge App SDK – A Scalable and Flexible Service The Edge App SDK is designed as a scalable and flexible tool for developing Edge apps, eliminating the need for Industrial Edge devices or management systems. It caters to both IT-centric app developers and those aiming to reduce development efforts by 30% through a streamlined Create-Test-Deploy workflow. Given below is an example use case for developing IIH apps without utilizing Industrial Edge. ## Core Workflows – Create, Test, and Deploy The SDK offers 3 major capabilities that not only ease up the app development process but also make it very fast and efficient. Given below is an example use case for developing edge apps using IIH dependencies. ### 1. Create Your App - App Code Generation - Faster development through code generation (templates, libraries). - Wraps the runtime dependencies and guides developers to add business logic in simple steps. ### 2. Test your app - Local Testing & Debugging - Efficient testing and debugging by emulating device services. - Enables automation of unit tests in the development environment. - Provides live metrics of dependency resource utilization. ### 3. Deploy your app - Build & Publish - Optimal packaging for delivery or deployment to IEM. - Speeds up overall development time and enhances the app development experience. ## IT Centric App Developer Journey ## Value The Edge App SDK significantly accelerates the edge application development process, offering several key benefits: 1. **Streamlined Development Workflow**: The SDK provides a standardized process for creating, testing, and deploying edge applications. This reduces the complexity and time required for each development stage. 1. **Boilerplate Code Generation**: The Create workflow generates boilerplate template code based on specified dependencies and services. This ensures consistency in application structure and allows developers to focus on custom logic rather than setup tasks. 1. **Simulation-Based Testing**: The Test/Emulate workflow allows developers to thoroughly test their applications in a simulated environment before deploying them to actual edge devices. This helps identify and resolve issues early in the development cycle, improving application reliability and performance. 1. **Efficient Deployment**: The Deploy workflow generates .app files that can be easily loaded onto the Industrial Edge Management (IEM) system. This facilitates the installation and management of edge applications on industrial devices. Overall, the SDK enhances the development experience by reducing repetitive tasks, minimizing errors, and speeding up the time to market for industrial edge applications. # Enlighted documentation # Contact For any technical questions related to the Enlighted APIs, please submit a request through the [Enlighted Support Portal](https://support.enlightedinc.com/support/s/login/), or on the [Enlighted Support site](https://support.enlightedinc.com/help/s/) you can use the chat button in the bottom right corner of each page. # Enlighted APIs - [Overview](overview.html) - \*/ - [Contact](contact.html) # Enlighted APIs The Enlighted lighting solution optimizes energy savings while enhancing occupant productivity, well-being, and security. The Enlighted APIs comprise - [Occupancy APIs](occupancy/overview.html) - [Location Services APIs](location-services/overview.html) - [Energy & Environment APIs](energy-environment/overview.html) ## Occupancy APIs Enlighted Occupancy APIs allow developers, students, or analytics teams to analyze information related to the presence of people in a building, their movement patterns, and other related metrics. Occupancy APIs can be used to access data for analyses in BI Tools or to operationalize data with other technology for automating real estate decisions or processes. Refer to Enlighted [Occupancy APIs](occupancy/overview.html). ## Location Services APIs Enlighted Location Intelligence leverages Bluetooth Low Energy (BLE) asset tags and the Enlighted lighting system for indoor asset tracking. These BLE tags communicate with Enlighted sensors, which use the received signal strength indicator to predict tag locations. This location prediction is powered by a refined deep learning model within the Enlighted Location Engine. The solution provides real-time insights into the location and movement of assets within a building. The data collected is managed in the Intelligent Workspace Intelligence Platform, supporting asset tracking applications and Business Intelligence (BI) reporting tools. Refer to Enlighted [Location Services REST APIs](location-services/overview.html). Refer to Enlighted [Location Services Streaming APIs](https://support.enlightedinc.com/help/s/article/ka03o000001DtGqAAK). ## Energy & Environment APIs Manage server stores sensor, status, and fault data from fixtures in the building, allowing users to configure fixture lighting behavior and monitor energy consumption and device status. The APIs allow users to monitor energy consumption, BACnet health and turn lights on during an emergency, obtain organization, floor, area, and fixture details. The Demand Response APIs contribute to energy load reduction and savings during peak demand time. Refer to Enlighted [Energy & Environment APIs](energy-environment/overview.html). # Changelog The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) ### 1.0.0 #### Converted REST API documentation to follow OpenAPI Specification # FAQ ## Temperature APIs ### Is a temperature API available? Yes. See [Get Sensor Details](api-reference.html). ### How accurate is the temperature data? The temperature data is accurate within two degrees Fahrenheit over the Sensor's operating range. The temperature sensor is inside the sensor. And the sensor is frequently inside a light fixture. Consequently, the sensor temperature is generally more helpful in detecting temperature changes rather than for absolute temperature measurement of the space below. ## Data Transfer ### How often does the data transfer, and is there any way to get it down to shorter (e.g., a few seconds)? The data transfer occurs every 5 minutes. No, you cannot shorten the time. ### Are the APIs exposed, accessible, and secure (Radio and Wired)? Data is received and transmitted over HTTPS. API authentication employs a hashed token linked with a single user. Refer to the [User Authentication](getting-started.html) for APIs article for more information. ### What are the conditions for accessing data and commands described in the APIs for call frequencies per minute/hour/days/month and for the supported data volume? Request call frequency should not exceed one request per 500ms. The parameter limits for a given API are documented in the description. Please refer to the APIs Documentation for more information. # Getting Started In Manage, users are assigned to a user role and a facility for greater control and security. When a user is assigned to a facility, the user can only issue API calls for the assigned facility and role permissions. A role grants the user the ability to perform specific tasks. You can manage roles and assign facilities to a user in the *User Management* section in the *Manage Administration* menu. ## Related Links - [User Role Permissions](https://support.enlightedinc.com/help/s/article/ka03o000001DtyiAAC) - [Add Users and User Roles](https://support.enlightedinc.com/help/s/article/ka03o000001DtvuAAC) in Manage - [Assign a Facility](https://support.enlightedinc.com/help/s/article/ka03o000001DtvjAAC) to a User When an API request is made, if the user role provides permission to the requested API and the user has access to the assigned facility, the request will be authorized and allowed to be completed. For example, users can request data from the sensors on the floor or area in the facility to which they have access. The API call returns a permission error if the user does not have access to the facility or permission to view data. ## API Authentication for Manage v4.5 and Above Users must be authenticated to send or receive API requests to and from Manage. For authentication, send the following headers along with the REST API. ### Generating the API Key To generate an API key for a user, see [Generate API Key](https://support.enlightedinc.com/help/s/article/ka03o000001DtpNAAS). For API authentication, send the following **Headers** along with the REST API call: - `UserId` – username - `Authorization` – Generated API key copied from the Manage application. For example, a user is assigned the following values: - UserId: `bob` - Authorization: `apikey e511c7a4b04740f2f3c519209ad7429ac3f9f728b97c5d8cd1c88096987ad0d1` ### API Example ```shell >curl -s --get -H "UserId: **bob**" -H "Authorization: apikey **e511c7a4b04740f2f3c519209ad7429ac3f9f728b97c5d8cd1c88096987ad0d1**" -H "Accept:  application/json" -k "http://localhost:8080/ems/api/switch-groups?facility=FLOOR&facilityId=5" -v -k *   Trying 127.0.0.1:8080... * Connected to localhost (127.0.0.1) port 8080 (#0) > GET /ems/api/switch-groups?facility=FLOOR&facilityId=5 HTTP/1.1 > Host: localhost:8080 > User-Agent: curl/7.83.1 > UserId: bob > Authorization: apikey e511c7a4b04740f2f3c519209ad7429ac3f9f728b97c5d8cd1c88096987ad0d1 > Accept:  application/json > ``` **Note**: Ensure that the API key and other credentials are entered in the request **Header**. ## API Authentication for Manage v4.4 and Below Users must be authenticated to send or receive API requests to and from Manage. For authentication, send the following headers along with the REST API. - `API key` – Unique identifier for the user (this is the user name, for example, Bob and the generated API key copied from Manage). - `Timestamp` - Time, date, and day of the API call. This is included to avoid replay attacks. - `Authorization` - SHA-1 authorization key (Calculated using the API key and timestamp). ### Generating the API Key To generate an API key for a user, see [Generate API Key](https://support.enlightedinc.com/help/s/article/ka03o000001DtpNAAS). Then, determine the timestamp (ts), authorization token as explained below, and send the headers along with the REST API call. For example, user Bob is assigned the following values: - Username: `bob` - API Key: `6eb6f07fd09b18dd61dd353dfb669820e7859cd3` (Copied from Manage) ### Time Stamp and SHA-1 Authorization Calculate timestamp and SHA-1 authorization values for the user as follows: 1. Use the formula below to calculate timestamp (ts): `ts=echo $(($(date +%s%N)/1000000))` For example, if today's GMT date and time is Thursday, March 3,  2016, 7:36:51.032 PM, the timestamp will be 1457033811032 `ts: 1457033811032` 1. Use the following command to calculate SHA-1 authorization in Linux: ```shell SHA1="$(echo -n "$username$apikey$ts" | sha1sum -t | awk '{print $1}')". ``` - Authorization: e20ac2c963ccfacf23a1f70287286443820e66d1 For API authentication, send the following Headers along with the REST API call: - APIkey: bob (Note: The APIkey here is the username) - Authorization: e20ac2c963ccfacf23a1f70287286443820e66d1 - ts: 1457033811032 ### API Example ```shell :~$ curl -s --get -H "ApiKey: bob" -H "Authorization: e20ac2c963ccfacf23a1f70287286443820e66d1" -H "ts:1457033811032" -H "Accept: application/xml" -H "Content-Type: application/xml" -k https://em_ip_address/ems/api/org/em/v1/energy -v -k * Hostname was NOT found in DNS cache * Trying 127.0.0.1... * Connected to localhost (127.0.0.1) port 443 (#0) * successfully set certificate verify locations: * CAfile: none CApath: /etc/ssl/certs * SSLv3, TLS handshake, Client hello (1): * SSLv3, TLS alert, Client hello (1): * SSLv3, TLS handshake, Server hello (2): * SSLv3, TLS handshake, CERT (11): * SSLv3, TLS handshake, Server key exchange (12): * SSLv3, TLS handshake, Server finished (14): * SSLv3, TLS handshake, Client key exchange (16): * SSLv3, TLS change cipher, Client hello (1): * SSLv3, TLS handshake, Finished (20): * SSLv3, TLS change cipher, Client hello (1): * SSLv3, TLS handshake, Finished (20): * SSL connection using ECDHE-RSA-AES256-GCM-SHA384 * Server certificate: * subject: C=US; ST=California; L=Sunnyvale; O=Enlighted Inc.; OU=Manage * start date: 2015-12-15 06:37:22 GMT * expire date: 2040-12-08 06:37:22 GMT * issuer: C=US; ST=California; L=Sunnyvale; O=Enlighted Inc.; OU=Manage * SSL certificate verify result: self signed certificate (18), continuing anyway. > GET /ems/api/org/facility/v1/energy/1 HTTP/1.1 > User-Agent: curl/7.35.0 > Host: localhost > ApiKey: bob > Authorization: e20ac2c963ccfacf23a1f70287286443820e66d1 > ts: 1457033811032 > Accept: application/xml > Content-Type: application/xml > < HTTP/1.1 200 OK < Date: Thu, 03 Mar 2016 19:42:55 GMT * Server Apache-Coyote/1.1 is not blacklisted < Server: Apache-Coyote/1.1 < Content-Type: application/xml < Content-Length: 234 < Set-Cookie: JSESSIONID=8CE1F42FCBCCEE5AB69175D45673951F; Path=/ems/; HttpOnly < Via: 1.1 127.0.0.1 < Vary: Accept-Encoding ``` # Energy & Environment APIs - [Overview](overview.html) - [Getting Started](getting-started.html) - [API Reference](api-reference.html) - [Changelog](changelog.html) - [FAQ](faq.html) # Energy and Environment APIs The Enlighted lighting solution optimizes energy savings while enhancing occupant productivity, well-being, and security. Manage server stores sensor, status, and fault data from fixtures in the building, allowing users to configure fixture lighting behavior and monitor energy consumption and device status. Manage supports standard REST-based APIs, including XML and JSON message formats. Developers can write applications using REST APIs to query and retrieve information from Manage. The sample responses to the web services are in JSON or XML format. Users must be authenticated to send or receive API requests to and from Manage. Learn about [User Authentication for APIs](getting-started.html). The Energy and Environmental APIs are complimentary with the purchase of [Connected Lighting or IoT](https://support.enlightedinc.com/help/s/article/Connected-Lighting-and-IoT-Sensor-Tiers) sensors and do not require additional licenses. The Occupancy APIs require [Sensor license](https://support.enlightedinc.com/help/s/article/Sensor-License) and [API license](https://support.enlightedinc.com/help/s/article/API-License). ## Value After user authorization, the APIs allow users to monitor energy consumption, BACnet health and turn lights on during an emergency, obtain organization, floor, area, and fixture details. The Demand Response APIs contribute to energy load reduction and savings during peak demand time. ## Common Request, Response Headers and Codes The Enlighted Manage (EM) supports standard REST-based APIs including XML and JSON messaging formats. Developers can write applications using REST APIs to query and retrieve information from Manage. The sample responses to the web services are in JSON or XML format. Refer to the [APIs Request, Response headers and Error Codes](https://support.enlightedinc.com/help/s/article/ka03o000001Du0oAAC). ## User Role Permissions API access affects both the objects that can be requested as well as the objects that are returned when making API requests. Follow the guidelines on API access by API endpoint and Manage role-based permissions. Refer to [User Role Permissions for APIs](https://support.enlightedinc.com/help/s/article/User-Role-Permissions-for-APIs). ## APIs - Organization APIs - Area APIs - Fixture APIs - Plugload APIs - Conference Room APIs - Data APIs - Demand Response APIs - Bluetooth Services (BLE) APIs - Tunable White APIs - DALI Emergency APIs # Changelog The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) ### 0.1.0 #### Converted REST API documentation to follow OpenAPI Specification # FAQ ## Location Services APIs ### What prerequisites are required for Location Intelligence? An Enlighted lighting system designed to meet Enlighted IoT design guidelines running on a cloud-based Energy Manager. The facility must be equipped with the latest Gen 5 sensor hardware. Location Intelligence is powered by AWS using off-the-shelf asset and badge tags with BLE capabilities to communicate with Enlighted sensors. ### How accurate is Location Intelligence? There is no indoor location system that can provide 100% accuracy, 100% of the time due to physics and technology constraints. Multiple factors affect the performance of the BRI system, such as: sensor density, building material composition, number of tags actively in use in each area, and RF interference. The room-level accuracy can increase to above 98% with 10-15s delay when rooms are well defined and have solid walls. The RF permeability of building materials, zone size and sensor density all have an impact on room-level accuracy. Unlike other RTLS systems which may experience degraded performance with thicker building materials, our machine-learning based algorithm with BLE fingerprinting benefits from solid walls. Reduced RF permeability helps identify the room with higher certainty. ### Is historical location data available? Both live time and one year of historical data are available on the Location Intelligence application, BlueGPS. Location Intelligence APIs ingested by third-party software provide live data only. ### What is a streaming API, and how does it work? A streaming API is a web service that provides a continuous flow of data in real-time. It allows clients to establish a connection and receive updates as soon as they occur. This is achieved through websocket or other streaming protocols, ensuring that clients are always up to date with the latest information. ### What are the benefits of using a streaming API over traditional polling requests? Streaming APIs eliminate the need for clients to repeatedly send requests to the server to fetch updates. Instead, the server pushes updates to connected clients as soon as they are available, reducing latency and network traffic. This is especially useful for time-sensitive applications. ### What are common use cases for streaming APIs in location-based services? Streaming APIs are valuable for various use cases, including asset tracking, space optimization, occupancy monitoring, and real-time location services. They enable applications to respond immediately to events like zone changes, button presses, or motion detections. ### How often should I refresh my credentials when using a streaming API? The frequency of refreshing credentials depends on the API and your specific use case. Some APIs may require more frequent updates than others. It's essential to review the API's documentation to determine the recommended refresh interval. ### Are there any best practices for optimizing the use of a streaming API in my application? Best practices include efficient use of filters, managing data subscriptions, handling errors gracefully, and ensuring your application can handle bursts of incoming data. Consult the API's documentation and consider load testing your application for robustness. # Getting Started The Location Intelligence APIs use Machine-to-Machine Authentication and Authorization with OAuth2.0 [client credentials flow](https://auth0.com/docs/get-started/authentication-and-authorization-flow/client-credentials-flow). End-user requests Enlighted for API access. Once approved, Enlighted will email the end-user with a Client ID and Client Secret. Request Auth0 service for a Bearer token using a Client ID and Secret provided by Enlighted. ## Example request for Bearer Token ```shell curl --request POST \ --url https://enlightedinc-p01.us.auth0.com/oauth/token \ --header 'content-type: application/json' \ --data '{"client_id":,"client_secret":,"audience":"https://rtls-us-east-1-prod.dsp.enlightedinc.com","grant_type":"client_credentials"}' ``` ### Response ```shell { "access_token": , "token_type": "Bearer" } ``` - [Overview](overview.html) - [API Reference](api-reference.html) - [Getting Started](getting-started.html) - [Changelog](changelog.html) - Location Services Streaming APIs - [Getting Started](location-services-streaming-apis/getting-started.html) - [Schema Overview](location-services-streaming-apis/schema-overview.html) - Data Types - [battery](location-services-streaming-apis/data-types/battery-data-type.html) - [button](location-services-streaming-apis/data-types/button-data-type.html) - [error](location-services-streaming-apis/data-types/error-data-type.html) - [floorExit](location-services-streaming-apis/data-types/floor-exit-data-type.html) - [positional](location-services-streaming-apis/data-types/positional-data-type.html) - [refreshSuccess](location-services-streaming-apis/data-types/refresh-success-data-type.html) - [tag](location-services-streaming-apis/data-types/tag-data-type.html) - Event Types - [attention](location-services-streaming-apis/event-types/attention-event-type.html) - [battery](location-services-streaming-apis/event-types/battery-event-type.html) - [floorExit](location-services-streaming-apis/event-types/floor-exit-event-type.html) - [getTagsFailure](location-services-streaming-apis/event-types/get-tags-failure-event-type.html) - [motion](location-services-streaming-apis/event-types/motion-event-type.html) - [positionChange](location-services-streaming-apis/event-types/position-change-event-type.html) - [requestError](location-services-streaming-apis/event-types/request-error-event-type.html) - [response](location-services-streaming-apis/event-types/response-event-type.html) - [tagHeartBeat](location-services-streaming-apis/event-types/tag-heart-beat-event-type.html) - [tagInfo](location-services-streaming-apis/event-types/tag-info-event-type.html) - [tamper](location-services-streaming-apis/event-types/tamper-event-type.html) - [zoneTransition](location-services-streaming-apis/event-types/zone-transition-event-type.html) - Command Types - [getTags](location-services-streaming-apis/command-types/get-tags-command-type.html) - [refreshCredentials](location-services-streaming-apis/command-types/refresh-credentials-command-type.html) - [FAQ](faq.html) # Location Services (RTLS) APIs ## Location Services REST APIs The Location Intelligence (LI) APIs, accessible through RESTful HTTP APIs, provide seamless interaction with asset and badge tags. These [tags](https://support.enlightedinc.com/help/s/topic/0TO3o000001dGH2GAM/li-spec-sheets) are tracked within the Location Intelligence system using their BLE capabilities and the Enlighted sensors. The APIs facilitate real-time tracking and monitoring of tag movement, triggering events based on configured conditions. Physical asset tags are equipped with attention and tamper buttons, that allow for events to be triggered from the tags. Users have the flexibility to configure Bluetooth tag settings directly through the APIs, allowing them to set reporting intervals and select sensors strategically to optimize network performance. The functionalities supported by the APIs include fetching individual tags, iterating over all tags, and conducting connectivity tests for a comprehensive user experience. ## Location Services Streaming APIs The Location Intelligence Streaming API uses real-time data retrieval by allowing clients to establish a connection that provides updates only when changes occur. Instead of relying on traditional polling requests, this API continuously queries and promptly delivers information as events unfold. This functionality is especially beneficial in scenarios like RTLS, where immediate awareness of events, such as button presses, is crucial. The Streaming API not only offers real-time data but also a diverse array of events, data types, and commands to facilitate seamless tracking and monitoring. By optimizing the management and tracking of tagged objects, this API becomes an invaluable resource for enhancing space utilization and efficiency. Features: - Monitor the real-time motion of tagged objects within a space. - Effortlessly access detailed data on battery status, location, and various attributes of tracked objects. - Harness the power of commands to request specific information, ensuring the status of connected devices. - Utilize secure connections and user authentication to protect data integrity and control access. - Leverage location optimization for improved decision-making based on real-time location data. # Getting Started The Streaming API uses the same Authentication & Authorization mechanisms as the RTLS API. Simply follow the instructions here and provide the same Authorization and x-dsp-selected-tenant-role headers when establishing the websocket connection. To ensure that a client’s credentials remain valid for the duration of the connection, clients will have to update their credentials at a regular cadence, or they will be disconnected. See the refreshCredentials command and refreshSuccess event for more details on how to keep the credentials up-to-date. The Location Intelligence APIs use Machine-to-Machine Authentication and Authorization with OAuth2.0 [client credentials flow](https://auth0.com/docs/get-started/authentication-and-authorization-flow/client-credentials-flow). End-user requests Enlighted for API access. Once approved, Enlighted will email the end-user with a Client ID and Client Secret. Request Auth0 service for a Bearer token using a Client ID and Secret provided by Enlighted. ## Example request for Bearer Token ```shell curl --request POST \ --url https://enlightedinc-p01.us.auth0.com/oauth/token \ --header 'content-type: application/json' \ --data '{"client_id":,"client_secret":,"audience":"https://rtls-us-east-1-prod.dsp.enlightedinc.com","grant_type":"client_credentials"}' ``` ### Response ```shell { "access_token": , "token_type": "Bearer" } ``` ## Connection parameters The following parameters can be provided as query string parameters when establishing the connection to the streaming API in order to control the connection: | Parameter | Type | Default | Notes | | ---------------------- | ------------------------------------------------------------------ | -------------------------------- | --------------------------------------------------------------------------------- | | `enterpriseId` | int | N/A | The ID of the enterprise to subscribe to data for. Must be provided | | `floors` | string (can be provided multiple times to select multiple options) | N/A | Set of floors to subscribe to data from. If not provided, no filtering is applied | | `eventTypes` | string (can be provided multiple times to select multiple options) | All event types except floorExit | The eventTypes to subscribe to | | `tagHeartbeatInterval` | int (must be between 300 and 86400) | 14395 | How frequently (in seconds) to receive tagHeartbeat messages | # Schema Overview | Property | Type | Description | | --------- | ------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | dataType | string | The schema that the payload of the event follows, specifying the structure and format of the data contained in the event. | | eventType | string | The type of event raised, indicating the nature of the occurrence. Examples include zone transitions, motion detection, or button presses. | | payload | object | The actual contents of the event, providing detailed information based on the event type. The payload may include specific data related to the event, such as tag information or positional data. | | requestId | string | Only present in events that are the result of a command. It provides a unique identifier to correlate the event to the source command. This is useful for tracking the origin of the event and its associated command. | # Command Type: getTags Description: The "getTags" command is used to retrieve information about tagged objects or devices that are being monitored by the streaming API. Use Case: This command gathers detailed information about tracked objects, including their attributes, status, and other relevant data. Response: [tagInfo](../event-types/tag-info-event-type.html) | Property | Type | Notes | | ----------- | --------------- | ------------------------------------------------------------------------- | | `type` | string | Must be `getTags` | | `tags` | list of string | Mac addresses to retrieve. If not specified, all known tags are retrieved | | `requestId` | optional string | If provided, responses to the command include the same requestId | # Command Type: refreshCredentials Description: The "refreshCredentials" command is a feature used to update and extend the authentication credentials to establish and maintain a connection with the streaming API. Use Case: This command ensures that the client's authentication credentials remain valid and up to date during the connection duration. It helps maintain a secure and uninterrupted connection to the streaming API. Response: [refreshSuccess](../data-types/refresh-success-data-type.html) | Property | Type | Notes | | -------------------- | --------------- | --------------------------------------------------------------------------------------------------------------------- | | `type` | string | Must be `refreshCredentials` | | `token` | string | Auth0 token to reauthenticate with. Must be `Bearer ` | | `selectedTenantRole` | string | Tenant to role mapping that indicates which role provided by the token to assume. Has the form `:` | | `requestId` | optional string | If provided, responses to the command include the same requestId | # Data Type: battery​ Context: The "battery" data type provides information about the battery status of tracked devices or tags within the system. Function: It allows you to monitor the battery life of devices, which is crucial for ensuring they remain operational. This data type reports details like battery voltage, level, and timestamps of updates. Use Case: You can use the "battery" data to proactively replace or recharge batteries as needed to prevent downtime and maintain reliable tracking. | Property | Type | Notes | | ------------------------- | ---------------------------------------------- | ----------------------------------------------------- | | `tagId` | UUID | | | `tagMac` | string (mac address - uppercase, no delimiter) | | | `floor` | string | | | `batteryVoltageTimestamp` | datetime | The timestamp the latest battery update was received | | `batteryLevel` | string | One of `CRITICAL`, `LOW`, `MEDIUM`, `GOOD` and `FULL` | | `batteryVoltage` | float | | | `schemaVersion` | int | Always `1` in this release | # Data Type: button Context: The "button" data type is associated with actions related to physical buttons on tracked objects. Function: It provides insights into button presses, including details about which button was pressed and when. This data type is beneficial for detecting actions like tamper alerts or user-initiated attention signals. Use Case: You can use the "button" data to respond to specific user interactions triggered by button presses. | Property | Type | Notes | | ----------------- | ---------------------------------------------- | ---------------------------------------------------- | | `tagId` | UUID | | | `tagMac` | string (mac address - uppercase, no delimiter) | | | `floor` | string | | | `buttonType` | string | One of `tamper` and `attention` | | `buttonTimestamp` | datetime | The timestamp the latest battery update was received | | `tamperState` | bool | Only present when `buttonType` is `tamper` | | `schemaVersion` | int | Always 1 in this release | # Data Type: error Context: The "error" data type is used to report and communicate error messages or issues. Function: It details the nature of errors, helping users and administrators identify and troubleshoot issues related to the system or API. Use Case: "error" data is crucial for diagnosing problems, allowing for timely resolution and system optimization. | Property | Type | Notes | | -------------- | ------ | ----- | | `errorMessage` | string | | # Data Type: floorExit Context: The "floorExit" data type is triggered when a tagged object leaves a particular floor. Function: It informs you when an object exits a specific floor area, with details about the floor and timestamps of the event. Use Case: "floorExit" data is valuable for monitoring and securing access to specific floors in multi-level facilities, ensuring that you are aware of any departures from designated areas. | Property | Type | Notes | | ---------------------- | ---------------------------------------------- | ----------------------------------------- | | `tagId` | UUID | | | `tagMac` | string (mac address - uppercase, no delimiter) | | | `floor` | string | The floor that was exited | | `computationTimestamp` | datetime | The time at which the floor exit computed | | `schemaVersion` | int | Always `1` in this release | # Data Type: positional Context: The "positional" data type focuses on the location and motion of tracked objects within a space. Function: It provides real-time data about the position, zone, motion status, and associated details for tagged objects. This data type is vital for monitoring the movement of assets or people within a defined area. Use Case: You can use "positional" data to track the flow of objects, detect zone transitions, and optimize the use of space in various applications. | Property | Type | Notes | | ---------------------- | ---------------------------------------------- | --------------------------------------- | | `tagId` | UUID | | | `tagMac` | string (mac address - uppercase, no delimiter) | | | `floor` | string | | | `zone` | string | | | `computationTimestamp` | datetime | The time at which the zone was computed | | `motion` | bool | Whether or not motion is detected | | `displayCoordinates` | object containing `x` and `y` properties | | | `schemaVersion` | int | Always `1` in this release | # Data Type: refreshSuccess Context: The "refreshSuccess" data type signifies a successful refresh of authentication credentials. Function: It confirms that the authentication credentials have been successfully updated, ensuring continued access to the system. Use Case: "refreshSuccess" data is important for maintaining a secure and uninterrupted connection to the streaming API by acknowledging the successful refresh of credentials. | Property | Type | Notes | | ------------ | -------- | ----------------------------------------- | | `status` | string | The string literal "Success." | | `expiryTime` | datetime | The time the credentials will expire next | # Data Type: tag Context: The "tag" data type provides comprehensive information about tracked objects, including their location and motion. Function: It offers a broad range of data, including position, zone, motion status, battery details, etc. This data type serves as a comprehensive data source for managing tagged objects. Use Case: You can use "tag" data to gain a holistic view of tagged objects, tracking their location, battery status, and motion behaviour, making it a versatile data type for various applications. | Property | Type | Notes | | ------------------------- | ---------------------------------------------------------------------- | ----------------------------------------------------------------- | | `tagId` | UUID | | | `tagMac` | string (mac address - uppercase, no delimiter) | | | `floor` | string | | | `zone` | string | | | `computationTimestamp` | datetime | The time at which the zone was computed | | `rawDataTimestamp` | datetime | The time at which the most recent raw data was received | | `motion` | bool | Whether or not motion is detected | | `displayCoordinates` | object containing `x` and `y` properties | | | `batteryVoltage` | optional float | | | `batteryLevel` | optional string | One of `CRITICAL`, `LOW`, `MEDIUM`, `GOOD`, and `FULL` if present | | `batteryVoltageTimestamp` | optional datetime | The time at which the most recent battery message was received | | `buttonStatus` | optional object containing `tamper` and `attention` boolean properties | | | `buttonTimestamp` | optional datetime | The time at which the most recent button message was received | | `schemaVersion` | int | Always `1` in this release | # Event Type: attention Triggered when the attention button is pressed on a tag. Description: The "attention" event type is triggered when a button on a tagged object is pressed to signal attention or a specific action. Use Case: “attention” events are typically used in applications where user-initiated actions or alerts need to be captured. Data Type: [button](../data-types/button-data-type.html) # Event Type: battery Triggered when the battery level of a tag changes (Ie transitions from `GOOD` to `MEDIUM`) or the voltage changes by more than 0.1. Description: The "battery" event type is triggered when there is a change in the battery level or voltage of a tagged object. It provides information about the battery status. Use Case: “battery” events are crucial for monitoring the power supply of tagged objects, ensuring they remain operational. Data Type: [battery](../data-types/battery-data-type.html) # Event Type: floorExit Triggered when a tag leaves a floor. Description: The "floorExit" event type occurs when a tagged object exits a particular floor within a multi-level facility. It provides information about which floor was exited. Use Case: “floorExit” events are valuable for monitoring access control and ensuring that objects do not leave designated areas. Data Type: [floorExit](../data-types/floor-exit-data-type.html) # Event Type: getTagsFailure Triggered when an unexpected error occurs while trying to execute a getTags command. Description: The "getTagsFailure" event type is triggered when an unexpected error occurs while trying to execute a "getTags" command. Use Case: “getTagsFailure” events report issues when retrieving tag information, allowing for troubleshooting and issue resolution. # Event Type: motion Triggered when a tag transitions from stationary to in motion, or vice versa. Description: The "motion" event type is generated when motion is detected. It informs you when an object starts or stops moving within the monitored area. Use Case: “motion” events are helpful for applications like occupancy monitoring, helping you detect movement and respond accordingly. Data Type: [positional](../data-types/positional-data-type.html) # Event Type: positionChange Triggered when the position of a tag changes. Description: The “positionChange” event type is triggered when the position of a tagged object changes. Use Case: “positionChange” events are important for tracking the movement of objects within a space and can be used in applications like space optimization and occupancy insights. Data Type: [positional](../data-types/positional-data-type.html) # Event Type: requestError Triggered in response to a malformed request. Description: The "requestError" event type is triggered in response to a malformed request or an issue with the command sent to the system. Use Case: “requestError” events help identify and address problems with requests, ensuring that the system operates smoothly. Data Type: [error](../data-types/error-data-type.html) # Event Type: response Triggered in response to a refreshCredentials command. Description: The "response" event type is generated in response to a "refreshCredentials" command. It provides information about the status of the credential refresh operation. Use Case: “response” events confirm the success or failure of credential updates, ensuring the continued security of the connection. Data Type: [error](../data-types/error-data-type.html) `|` [refreshSuccess](../data-types/refresh-success-data-type.html) # Event Type: tagHeartBeat Description: Triggered by the first position update received from a tag after tagHeartbeatInterval seconds have passed since the last tagHeartbeat, regardless of whether anything has changed. Use Case: “tagHeartbeat” events help maintain regular communication with tagged objects and ensure that their status is up to date. Data Type: [positional](../data-types/positional-data-type.html) # Event Type: tagInfo Triggered in response to a getTags command. Description: The "tagInfo" event type is generated in response to a "getTags" command. It provides detailed information about tagged objects, including their attributes and status. Use Case: “tagInfo” events retrieve comprehensive data about tagged objects for management and tracking purposes. Data Type: [tag](../data-types/tag-data-type.html) # Event Type: tamper Triggered when a tag enters a tamper state, or leaves it. Description: The "tamper" event type is generated when a tagged object enters a tamper state or exits it. Tamper events are related to the integrity of the object. Use Case: “tamper” events are essential for detecting and responding to potential tampering or unauthorized access to tagged objects. Data Type: [button](../data-types/button-data-type.html) # Event Type: zoneTransition Triggered when a tag’s zone changes. Description: The "zoneTransition" event type is triggered when a tagged object moves from one zone to another within a defined space. It indicates a change in the location or zone of the object. Use Case: “zoneTransition” events are important for tracking the movement of objects within a space and can be used in applications like space optimization and occupancy insights. Data Type: [positional](../data-types/positional-data-type.html) # Changelog The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) ### 1.0.0 #### Converted REST API documentation to follow OpenAPI Specification # FAQ ## Occupancy APIs ### Q: How long is Occupancy API data available? : **A:** For the Fine-Grain Motion, Occupancy API, data is retained with 5 min granularity for 45 days. If you need to store occupancy data for a longer period, consider the [Space](https://support.enlightedinc.com/help/s/topic/0TO3o000001dGBJGA2/space) offering. The real-time data from Area APIs is not stored in the database. ### Q: What is the granularity and delay of the occupancy data provided by the sensors through the API? : **A:** The Occupancy API data provided by the sensors is in real-time, with a delay of only 5 minutes. The Fine-Grain Motion API provides occupancy information for the last 5 minutes, with a granularity of 5 seconds at the time when the sensor last reported. ### Q: How far back is the occupancy data available for the Space application? Does the granularity change over time? : **A:** If the Energy Manager (EM) server is located on-site, you can access Occupancy data in the [Space](https://support.enlightedinc.com/help/s/topic/0TO3o000001dGBJGA2/space) application from up to 45 days ago. However, if the EM is connected to eCloud or EMC (Energy Manager in the Cloud), you can access Occupancy data from the time the system was first deployed. ### Q: Will a RestAPI call to Manage provide the most recent area (zonal) occupancy status from the Manage database? : **A:** Yes. When an area is enabled as a [Zonal area](https://support.enlightedinc.com/help/s/article/ka03o000001DtwIAAS) and [API License (EM-API-OCC)](https://support.enlightedinc.com/help/s/article/API-License) has been purchased based on the number of sensors configured in the system, you can expect sub-minute end-to-end latency. ### Q: Does placing a sensor into an Area and enabling Zonal Occupancy enable the sensor to start reporting occupancy events immediately? : **A:** Yes. The sensor reports area occupancy status. Refer to [Enable Zonal area](https://support.enlightedinc.com/help/s/article/ka03o000001DtwIAAS) to enable sensors in an area to report occupancy. ### Q: If Zone Sensor is enabled, will a zoned sensor report an occupancy event immediately to Manage server? Will it report a vacancy event 90 seconds after the most recent occupancy detection? : **A:** Yes. The sensor sends Area Occupancy Status to Manage by Real-Time packet, sent when the Area Occupancy Status: occupied (1) or unoccupied (0) event occurs. When the event occurs, a counter of 90 seconds starts on the sensor. An [API License (EM-API-OCC)](https://support.enlightedinc.com/help/s/article/API-License) is required for the sensor to send real-time occupancy. ### Q: If a sensor was placed into an Area, but Zonal Occupancy was not enabled, will the sensor still report Occupancy status? : **A:** No. ### Q: Does a zonal sensor only report the first occupancy event in the vacancy state, or does it report each time it detects motion? : **A:** The sensor will report Occupancy Status via Real-Time packet every 90 sec as long as the sensor continuously detects motion. # Getting Started In Manage, users are assigned to a user role and a facility for greater control and security. When a user is assigned to a facility, the user can only issue API calls for the assigned facility and role permissions. A role grants the user the ability to perform specific tasks. You can manage roles and assign facilities to a user in the *User Management* section in the *Manage Administration* menu. ## Related Links - [User Role Permissions](https://support.enlightedinc.com/help/s/article/ka03o000001DtyiAAC) - [Add Users and User Roles](https://support.enlightedinc.com/help/s/article/ka03o000001DtvjAAC) in Manage - [Assign a Facility](https://support.enlightedinc.com/help/s/article/ka03o000001DtvuAAC) to a User When an API request is made, if the user role provides permission to the requested API and the user has access to the assigned facility, the request will be authorized and allowed to be completed. For example, users can request data from the sensors on the floor or area in the facility to which they have access. The API call returns a permission error if the user does not have access to the facility or permission to view data. ## API Authentication for Manage v4.5 and Above Users must be authenticated to send or receive API requests to and from Manage. For authentication, send the following headers along with the REST API. ### Generating the API Key To generate an API key for a user, see [Generate API Key](https://support.enlightedinc.com/help/s/article/ka03o000001DtpNAAS). For API authentication, send the following **Headers** along with the REST API call: - `UserId` – username - `Authorization` – Generated API key copied from the Manage application. For example, a user is assigned the following values: - UserId: `bob` - Authorization: `apikey e511c7a4b04740f2f3c519209ad7429ac3f9f728b97c5d8cd1c88096987ad0d1` ### API Example ```shell >curl -s --get -H "UserId: **bob**" -H "Authorization: apikey **e511c7a4b04740f2f3c519209ad7429ac3f9f728b97c5d8cd1c88096987ad0d1**" -H "Accept:  application/json" -k "http://localhost:8080/ems/api/switch-groups?facility=FLOOR&facilityId=5" -v -k *   Trying 127.0.0.1:8080... * Connected to localhost (127.0.0.1) port 8080 (#0) > GET /ems/api/switch-groups?facility=FLOOR&facilityId=5 HTTP/1.1 > Host: localhost:8080 > User-Agent: curl/7.83.1 > UserId: bob > Authorization: apikey e511c7a4b04740f2f3c519209ad7429ac3f9f728b97c5d8cd1c88096987ad0d1 > Accept:  application/json > ``` **Note**: Ensure that the API key and other credentials are entered in the request **Header**. ## API Authentication for Manage v4.4 and Below Users must be authenticated to send or receive API requests to and from Manage. For authentication, send the following headers along with the REST API. - `API key` – Unique identifier for the user (this is the user name, for example, Bob and the generated API key copied from Manage). - `Timestamp` - Time, date, and day of the API call. This is included to avoid replay attacks. - `Authorization` - SHA-1 authorization key (Calculated using the API key and timestamp). ### Generating the API Key To generate an API key for a user, see [Generate API Key](https://support.enlightedinc.com/help/s/article/ka03o000001DtpNAAS). Then, determine the timestamp (ts), authorization token as explained below, and send the headers along with the REST API call. For example, user Bob is assigned the following values: - Username: `bob` - API Key: `6eb6f07fd09b18dd61dd353dfb669820e7859cd3` (Copied from Manage) ### Time Stamp and SHA-1 Authorization Calculate timestamp and SHA-1 authorization values for the user as follows: 1. Use the formula below to calculate timestamp (ts): `ts=echo $(($(date +%s%N)/1000000))` For example, if today's GMT date and time is Thursday, March 3,  2016, 7:36:51.032 PM, the timestamp will be 1457033811032 `ts: 1457033811032` 1. Use the following command to calculate SHA-1 authorization in Linux: ```shell SHA1="$(echo -n "$username$apikey$ts" | sha1sum -t | awk '{print $1}')". ``` - Authorization: e20ac2c963ccfacf23a1f70287286443820e66d1 For API authentication, send the following Headers along with the REST API call: - APIkey: bob (Note: The APIkey here is the username) - Authorization: e20ac2c963ccfacf23a1f70287286443820e66d1 - ts: 1457033811032 ### API Example ```shell :~$ curl -s --get -H "ApiKey: bob" -H "Authorization: e20ac2c963ccfacf23a1f70287286443820e66d1" -H "ts:1457033811032" -H "Accept: application/xml" -H "Content-Type: application/xml" -k https://em_ip_address/ems/api/org/em/v1/energy -v -k * Hostname was NOT found in DNS cache * Trying 127.0.0.1... * Connected to localhost (127.0.0.1) port 443 (#0) * successfully set certificate verify locations: * CAfile: none CApath: /etc/ssl/certs * SSLv3, TLS handshake, Client hello (1): * SSLv3, TLS alert, Client hello (1): * SSLv3, TLS handshake, Server hello (2): * SSLv3, TLS handshake, CERT (11): * SSLv3, TLS handshake, Server key exchange (12): * SSLv3, TLS handshake, Server finished (14): * SSLv3, TLS handshake, Client key exchange (16): * SSLv3, TLS change cipher, Client hello (1): * SSLv3, TLS handshake, Finished (20): * SSLv3, TLS change cipher, Client hello (1): * SSLv3, TLS handshake, Finished (20): * SSL connection using ECDHE-RSA-AES256-GCM-SHA384 * Server certificate: * subject: C=US; ST=California; L=Sunnyvale; O=Enlighted Inc.; OU=Manage * start date: 2015-12-15 06:37:22 GMT * expire date: 2040-12-08 06:37:22 GMT * issuer: C=US; ST=California; L=Sunnyvale; O=Enlighted Inc.; OU=Manage * SSL certificate verify result: self signed certificate (18), continuing anyway. > GET /ems/api/org/facility/v1/energy/1 HTTP/1.1 > User-Agent: curl/7.35.0 > Host: localhost > ApiKey: bob > Authorization: e20ac2c963ccfacf23a1f70287286443820e66d1 > ts: 1457033811032 > Accept: application/xml > Content-Type: application/xml > < HTTP/1.1 200 OK < Date: Thu, 03 Mar 2016 19:42:55 GMT * Server Apache-Coyote/1.1 is not blacklisted < Server: Apache-Coyote/1.1 < Content-Type: application/xml < Content-Length: 234 < Set-Cookie: JSESSIONID=8CE1F42FCBCCEE5AB69175D45673951F; Path=/ems/; HttpOnly < Via: 1.1 127.0.0.1 < Vary: Accept-Encoding ``` - [Overview](overview.html) - [Getting Started](getting-started.html) - [API Reference](api-reference.html) - [Changelog](changelog.html) - [FAQ](faq.html) # Occupancy APIs Occupancy APIs support real-time space management and space utilization use cases. They require a Sensor License and API License based on the number of sensors configured in the system. Customers must purchase and install the licenses to remain in compliance. Refer to the [License Types and Product codes](https://support.enlightedinc.com/help/s/article/ka03o000001DtY6AAK) for more information. With these APIs, users can seamlessly retrieve occupancy data as occupied vs. unoccupied for an area, all areas on a floor, and detect whether a desk is occupied or unoccupied. In addition, users can obtain movement data of people within a building or space based on sensor triggered events at every five-minute interval. Granular data such as power and temperature is also available with the fine-grain motion API. Things to know: 1. Users must purchase Sensor License and API License. 1. Sensors in an area must be activated as zonal sensors to report occupancy data. Read [Enable Zonal Sensors](https://support.enlightedinc.com/help/s/article/ka03o000001DtwIAAS). 1. Desk sensors report occupancy status only when the sensor and Facility are assigned to 'Desk Sensor' Fixture Type. Read [Enable Desk Sensors](https://support.enlightedinc.com/help/s/article/Enable-Desk-Sensors-Surface-Sensor-SU-5i-USB-to-Report-Occupancy) (Surface Sensor, SU-5i-USB). # Electrification X documentation For any questions related to Electrification X, please contact **SI EA Global Support**\ Email: [support.ea.si@siemens.com](mailto:support.ea.si@siemens.com) Getting started with using Electrification X APIs involves the following steps: 1. Get credentials for authorization 1. Create a JSON Web Token (JWT) by using the credentials 1. Make API requests using the JWT ## Get credentials Contact your company administrator. Your company administrator will provide you with the following credentials: - Client id - Client secret ## Create a token For the Electrification X API you need a client id and a client secret to get an access token. ### Example request ```python import requests def authenticate(client_id: str, client_secret: str) -> str: response = requests.post('https://siemens-bt-015.eu.auth0.com/oauth/token', payload={ 'client_id': client_id, 'client_secret': client_secret, 'audience': 'https://horizon.siemens.com', 'grant_type': 'client_credentials', }) response.raise_for_status() message = reponse.json() return message['access_token'] ``` ### Example response ```json { "access_token": "eyJ0eXAiOiUSJ9.eyJpc3MiOiJdGlhbHMifQ.MJpcxLfyOt", "token_type": "Bearer", "expires_in": 86400 } ``` The access token is a JWT (JSON Web Token). It is the value of the `access_token`-property in the response. You can now use it by passing it in the `Authorization`-header of any subsequent API requests. The `expires_in`-property represents the number of seconds your token is valid. Usually, the value corresponds to 24 hours. When this time has elapsed you will need to create a new token. - [Overview](overview.html) - [Getting Started](getting-started.html) - [Electrification X API](docs/api-overview.html) - [API Reference](api_specs/api_reference.html) - [Setting up a machine user](docs/creating_machine_user.html) - [Contact](contact.html) # Electrification X **Electrification X** combines the real and digital worlds in the Xcelerator IoT Software as a Service (SaaS) offering for Electrification & Automation to tackle the challenges of energy transition. Electrification X helps renewable energy operators, transmission system operators (TSOs), distribution system operators (DSOs), Data Centers, Industries & Infrastructure customers to manage their energy networks, increase uptime and improve reliability, asset utilization, cybersecurity, and energy efficiency to reach sustainability goals. With its [**open APIs**](../apis.html?productLine=Electrification+X), Electrification X allows you to use its contextualized data to **build your own applications** or integrate the data into your IT tools which makes it an ideal solution for various stakeholders and users, from people who want a substation overview all the way to specialists that need specific details regarding dedicated assets and components. # Electrification X API # Electrification X API overview ## Navigation - [Entity relationship diagram](#entity-relationship-diagram) - [Hierarchical structure of entities in Electrification X](#hierarchical-structure-of-entities-in-electrification-x) - [Available entities](#available-entities) - [User](#user) - [Customer](#customer) - [Partition](#partition) - [Location](#location) - [Assets](#assets) - [Measurements](#measurements) - [MeasurementValue](#measurementvalue) - [Aggregate](#aggregate) ## Entity relationship diagram ``` classDiagram direction LR class User class Customer class Partition class Location class Asset class Measurement class MeasurementValue class Aggreate User "N" --> "1" Customer Customer "1" --> "N" Partition Partition "1" --> "N" Location Location "1" --> "N" Asset Location "1" --> "N" Measurement Asset "1" <-- "N" Measurement Measurement "1" --> "N" MeasurementValue Aggreate --|> MeasurementValue ``` ## Hierarchical structure of entities in Electrification X The entities in Electrification X follow a hierarchical structure: ## Available entities ### User In Electrification X, a user is a defined entity with specific access rights, roles, and permissions. ### Customer In Electrification X, a customer is a broader entity that represents the organization or entity utilizing Electrification X platform. ### Partition In Electrification X, a partition is a specific defined scope of work, tasks, or operational domain that an individual or team is accountable for. ### Location In Electrification X, a location is a specific site, facility, or geographical point, e.g. a station. ### Assets In Electrification X, an asset is electrical infrastructure and equipment critical for energy distribution and electrification. An asset is a logical unit which can consist of multiple physical devices. Examples for assets are: - Transformers - Circuit breakers - Switchgear - Feeders - EV charging stations Assets are represented in a hierarchical structure within the Electrification X application. In the asset hierarchy, measurement data can be accessed by traversing the asset structure, with each asset providing access to its associated measurement values. ### Measurements In Electrification X, a measurement is a data point that represents a quantitative observation or reading of a parameter, e.g. the ambient humidity of a station. ### MeasurementValue The MeasurementValue is the aggregated value of a measurement over the specified time period. ### Aggregate The values are aggregated for the specified start time, end time, and interval. You can select an interval of 15 minutes. # Setting up a machine user In the **Accounts** application, you can create machine users. While you create machine users, the Client ID and Client secret key can be copied and sent to a user for further usage of the API. ## Navigation - [Creating a machine user](#creating-a-machine-user) - [Creating a machine user group](#creating-a-machine-user-group) - [Assigning a machine user group to a partition](#assigning-a-machine-user-group-to-a-partition) - [Adding machine users to a machine user group](#adding-machine-users-to-a-machine-user-group) ## Requirements - You have the role **Machine User Administrator** ## Creating a machine user 1. Navigate to **User management > Machine users**.\ The page **Machine users** opens. 1. Select the button **+Create**.\ The dialog **Create machine user** opens. 1. Enter a name for the machine user. 1. Select a user group. 1. Enter a description. 1. Select the button **Create**.\ The machine user is created and the Client ID and Client secret key are generated. 1. Copy the Client ID and Client secret key. Select the button **Copy details** to copy the Client ID and Client secret key at once. > Copy the Client secret key within the dialog **Create machine user**, as this is the only time you can see it. If you don't copy the Client secret key and close the dialog, you need to create another one later. Siemens recommends to store the Client ID and Client secret key in a safe location, for example, by using a password manager. 1. Select **Save** to close the dialog. ## Creating a machine user group 1. Navigate to **User management > User groups**.\ The page **User groups** opens. 1. Select the button **+Create**.\ The dialog **Create group** opens. 1. Select the option **Machine user group**. 1. Enter a name for the machine user group. 1. Enter a description. 1. Activate the check boxes for the desired roles. 1. Select the button **Add**.\ The machine user group is created. ## Assigning a machine user group to a partition 1. In the page **User groups**, select the desired machine user group. 1. Switch to the tab **Partition**.\ The available partitions are shown. 1. Activate the check boxes of the desired partitions. 1. Select the button **Save**.\ The machine user group is assigned to the selected partitions. ## Adding machine users to a machine user group 1. In the page **User groups**, select the desired machine user group. 1. Switch to the tab **Users**.\ The already assigned machine users are shown. 1. Select the link **Manage users**.\ The page **Manage users** opens. 1. Select the machine users you want to add to the group on the left side of the page. 1. Select the right-arrow icon to add the selected machine users to the **User group members** list. 1. Select the button **Next**.\ The page **Manage users** closes and the selected machine users are added to the list. 1. Select the button **Save**. The selected machine users are added to the selected machine user group. # gWAP documentation # Contact gPROMS Web Applications Publisher is now available in [Support Center](https://support.sw.siemens.com/), the support portal for all Siemens Digital Industries Software products. Users are welcome to visit the portal to receive product updates, access knowledge base articles and documentation, open support cases, obtain license/order information, and much more. Email: [gwap.api.support.industry@siemens.com](mailto:gwap.api.support.industry@siemens.com) # Getting Started The API is intended to be used for machine-to-machine communication and system integration. ## Access tokens Authentication with the API needs to be done using an access token. To issue a new token, navigate to the "Access token" dialog using the button on the bottom-left corner. Here, all active access tokens can be viewed, and new ones issued. When issuing a new token, the following fields must be specified: - **Name**: A human-readable name used to identify the token. - **Expiration date**: The date after which the access token will automatically stop working. - **Scopes**: Permissions that the token will have. These take the format of `_`. Where an action is either `read` or `write`. Note Access tokens inherit all role- and application- permissions from the user who issued it. When used, the token acts on behalf of that user. Immediately after creation of an access token, its value will be displayed. Make sure to save it somewhere safe. After you leave the page, you no longer have access to the token. The token takes the form of a JSON Web Token (JWT) and needs to be included on all requests in a HTTP header: ```text Authorization: Bearer ``` Warning Access tokens must be treated carefully. Do not store tokens in plaintext in your projects. Use minimal scopes if possible, and set a short expiration date fitting your use-case. If a token is no longer required, or if it has been compromised, it can be revoked. This will immediately block any requests being authenticated with that token. - [Overview](overview.html) - [Get Started](getting-started.html) - [gWAP REST API](rest-api-spec.html) - [Contact](contact.html) # Overview gPROMS Digital Process Twins capture fundamental knowledge about a process. The model is then used in conjunction with state-of-the-art mathematical techniques to analyze and optimize the process design or operation – rapidly, accurately and effectively. Optimizing a process or product design can lock in value over the lifetime of production – amounting to billions of dollars in some cases. Optimizing a plant operation can generate new value on a daily basis. [Additional information on gPROMS](https://www.siemens.com/global/en/products/automation/industry-software/gproms-digital-process-design-and-operations.html) ## Introduction The gPROMS Web Applications Publisher (gWAP) can be used to deploy these complex models to an easy-to-use, easy-to-access, web application. This enables: - **Democratization of modeling and simulation** – Empowers non-modeling communities (R&D, manufacturing, and operations) to innovate, reduce costs, and boost productivity by leveraging process digital twins and model-based decision-making. - **Improved accessibility and usability** – Intuitive, browser-based UI makes predictive modeling easily available to non-experts, fostering trust and adoption of modeling solutions. - **Standardization and consistency** – Centralized system ensures process models are shared and used consistently across multiple teams and locations. - **Stronger governance and security** – Role-Based Access Control (RBAC) provides permission management, tracking, and enforcement of authorization policies. # gWAP REST API - [Download OpenAPI Specification](gwap-public-api.yaml) # Industrial AI Suite documentation # Contact How we can support you – let's connect and talk You are faced with the task of using Industrial AI solutions to increase process efficiency while maintaining profitability. Do you need a partner who takes care of shop floor integration and solution operation? Let's find out where and how we can provide you with the most valuable support – to help you achieve your objectives. [Get in contact](https://www.siemens.com/global/en/products/automation/topic-areas/industrial-ai/industrial-ai-suite.html#Howwecansupportyouletsconnectandtalk) - [Overview](overview.html) - Industrial AI Suite APIs - AI Asset Manager - [Overview](asset-manager/overview.html) - Async APIs - [Overview](asset-manager/async-apis/overview.html) - [AWS Cloud Package Delivery API](asset-manager/async-apis/aws-delivery-api/api.html) - [Azure Cloud Package Delivery API](asset-manager/async-apis/azure-delivery-api/api.html) - [Cloud API](asset-manager/async-apis/cloud-api/api.html) - OpenAPIs - [Overview](asset-manager/open-apis/overview.html) - [V2](asset-manager/open-apis/swagger_v2/api.html) - [V3](asset-manager/open-apis/swagger_v3/api.html) - AI Inference Server - [Overview](inference-server/overview.html) - [OpenAPI](inference-server/openapi/api.html) - AI SDK - [Overview](sdk/overview.html) - API - [Overview](sdk/docs/overview.html) - simaticai - [AI Software Development Kit CLI](sdk/docs/simaticai/__main__.html) - Modules - [common](sdk/docs/simaticai/common.html) - data - [data and schemas](sdk/docs/simaticai/data.html) - [Consent Information](sdk/simaticai/data/consent_info.html) - [deploy](sdk/docs/simaticai/deploy.html) - [deployment](sdk/docs/simaticai/deployment.html) - [helpers](sdk/docs/simaticai/helpers.html) - [model_config_pb2](sdk/docs/simaticai/model_config_pb2.html) - [packaging](sdk/docs/simaticai/packaging.html) - [payloads](sdk/docs/simaticai/payloads.html) - [telemetry](sdk/docs/simaticai/telemetry.html) - [Testing](sdk/docs/simaticai/testing.html) - log_module - [log_module](sdk/docs/log_module/log_module.html) - [mock_logger](sdk/docs/log_module/mock_logger.html) - Vision Connector App - [Overview](vision-connector-app/overview.html) - [OpenAPI](vision-connector-app/openapi/api.html) - Vision Data Collector - [Overview](vision-data-collector/overview.html) - [OpenAPI](vision-data-collector/openapi/api.html) - [Contact](contact.html) # Industrial AI Suite ## Why build on Siemens Industrial AI Suite? - Part of the Industrial Edge Ecosystem We prevent you from reinventing the wheel and wasting resources on building your own industrial AI tooling for deploying, running and monitoring AI solutions. - Bring your AI Model to the shopfloor All applications are designed to be user-friendly, allowing automation engineers with no prior data science experience to easily deploy, run, and monitor your AI solutions. - Use partnership for innovation All applications are natively integrated into our SIMATIC, Industrial Edge, and hardware portfolio, offering standard connectors for seamless compatibility. ## Scale with Industrial AI Suite Combine your AI expertise with our robust Industrial Edge portfolio for scalable and reliable solutions. Tailored to your tools and cloud provider, our suite simplifies the AI lifecycle – from model packaging and deployment to monitoring. Free your data scientists while we handle deployment, connectivity, inference and monitoring seamlessly. ## Seamless deployment of Industrial AI models onto the shop floor With our Industrial AI portfolio, you gain access to a toolkit that facilitates comfortable deployment of AI models on the shop floor, while integration with the Siemens Industrial Edge ecosystem takes care of data connectivity aspects. The combination of Siemens Industrial Edge and Industrial AI infrastructure simplifies and streamlines the implementation of AI solutions across various locations and production lines, resulting in a faster return on investment. ## Available APIs | Link | Description | | ------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | [AI Asset Manager](asset-manager/overview.html) | AI Asset Manager lets you keep track of all AI model deployments on Edge. Configure automated model downloads and deployment from cloud of AI providers to ease and speed up the integration of AI Models on the shop floor. Check if your AI Models meet your expectations on the shopfloor. | | [AI Inference Server](inference-server/overview.html) | AI Inference Server is an Industrial Edge App, which activates the IE Devices by embedding the inference function implemented in Python. | | [AI SDK](sdk/overview.html) | The AI Software Development Kit, or AI SDK for short, is an essential part of Siemens' Industrial Edge ecosystem. It is a set of Python libraries that provide building blocks for automating the creation, packaging, and testing of inference pipelines for the AI Inference Server. | | [Vision Connector App](vision-connector-app/overview.html) | The Industrial Edge Vision Connector App (VCA) combines industrial cameras with the Industrial Edge ecosystem. The Vision Connector uses an Ethernet interface to establish a connection to the cameras. The VCA publishes the data using the ZeroMQ server integrated in the Vision Connector with high throughput to other applications. | | [Vision Data Collector](vision-data-collector/overview.html) | Vision Data Collector is an Industrial Edge App that collects data, including images and metadata from Vision Connector, and inference results from pipeline packages deployed in AI Inference Server. | # AI Asset Manager ## Do you see AI as a potential solution for your current challenges? Are you concerned about how to manage a large number of AI models on the shop-floor, managing when and how they are deployed and updated in your production environments? Have you ever felt you need more data about your AI model execution? AI Asset Manager is the Edge Application to support AI model management on Industrial Edge, easing model download, tracking and deployment over a large number of devices. It enables customers to upload models via the UI, via API or download them automatically from the cloud (AWS Sagemaker, AzureML, others). Moreover, AI Asset Manager provides comprehensive supervision of your AI executions by collecting, storing and aggregating relevant metrics from multiple sources. ## Key features and benefits - Key features: - Automated deployment from the cloud to the Edge - Deployment of trained models to multiple Industrial Edge devices - Orchestration of deployed models - Tracking of AI Inference Server pipeline status - Edge Infrastructure monitoring to understand resource constraints - AI Model monitoring to validate model performance via custom metrics - Control actions on running deployments on AI Inference Server (e.g. start, stop) - Benefits - Reduced time and effort to deploy models to Industrial Edge using standard CI/CD tools and pipelines - Manual import of AI models from user's computer or cloud file storage possible - Maintain an overview of available devices and model deployments - Visualize all deployments from a single management dashboard - Keep track of inference performance, detecting HW and SW issues - Receive automated alerts whenever your model or data are not behaving as expected ## Available APIs | Async APIs | | -------------------------------------------------------------------------- | | [AWS Cloud Package Delivery API](async-apis/aws-delivery-api/api.html) | | [Azure Cloud Package Delivery API](async-apis/azure-delivery-api/api.html) | | [Cloud API](async-apis/cloud-api/api.html) | | OpenAPI versions | | ----------------------------------- | | [V2](open-apis/swagger_v3/api.html) | | [V3](open-apis/swagger_v3/api.html) | # Overview of AsyncAPIs | Async APIs | | --------------------------------------------------------------- | | [AWS Cloud Package Delivery API](aws-delivery-api/api.html) | | [Azure Cloud Package Delivery API](azure-delivery-api/api.html) | | [Cloud API](cloud-api/api.html) | # AI Asset Manager AWS Cloud Package Delivery API [Download Async API Specification](aws-delivery-api.yml) View the AsyncAPI specification # AI Asset Manager Azure Cloud Package Delivery API [Download Async API Specification](azure-delivery-api.yml) View the AsyncAPI specification # AI Asset Manager Cloud API [Download Async API Specification](cloud-api.yml) View the AsyncAPI specification # Overview of OpenAPIs - Dev Portal | OpenAPI versions | | ------------------------- | | [V2](swagger_v2/api.html) | | [V3](swagger_v3/api.html) | # AI Asset Manager API V2 [Download OpenAPI Specification](public_v2_api_swagger.yaml) View the OpenAPI specification # AI Asset Manager API V3 [Download OpenAPI Specification](public_v3_api_swagger.yaml) View the OpenAPI specification # AI Inference Server Reduce the cost, effort and time to bring AI models to the shop floor and help to focus on your domain expertise Do you see AI as a potential solution for your current challenges? But are you concerned about the high costs and effort to bring AI to the shop floor and maintain it? Are you interested in standardizing the execution of AI Models on the shop floor and integrate it with monitoring, debugging, and remote deployment capabilities? AI Inference Server is the edge application to standardize AI model execution on Siemens Industrial Edge. The application eases data ingestion, orchestrates data traffic, and is compatible all powerful AI frameworks thanks to the embedded Python interpreter. It enables the AI model deployment as content to a standard edge app instead of custom container deployment for single models. AI finally meets Siemens Industrial Edge and AI Inference Server leverages the benefits of both. ## Key Features and Benefits - Key Features - Supports most popular AI frameworks compatible with Python - Orchestrates and controls AI model execution - Ability to run AI pipelines both with an older and a newer version of Python - Allows horizontal scaling of AI pipelines for optimal performance - Simplifies tasks such as input mapping (thanks to the integration with Databus and other Siemens Industrial Edge connectors), data ingestion, pipeline visualization - Monitoring and debugging of AI models using inference statistics, logging and image visualization - Contains pipeline version handling - Import models via UI or receive them remotely - Supports persistent data storage on the local device for each pipeline - Benefits - Standardize the AI model execution using an AI-ready inference with the Edge ecosystem - Standardize the logging, monitoring, and debugging of AI models - Designed for MLOps Integration with AI Model Monitor ## Available APIs | OpenAPIs | | ------------------------------------------------ | | [Vision Connector App OpenAPI](openapi/api.html) | # AI Inference Server OpenAPI [Download OpenAPI Specification](openapi.json) View the OpenAPI specification # AI Software Development Kit (AI SDK) The AI Software Development Kit, or AI SDK for short, is an essential part of Siemens' Industrial Edge ecosystem. It is a set of Python libraries that provide building blocks for automating the creation, packaging, and testing of inference pipelines for the AI Inference Server. The AI SDK is accompanied by end-to-end tutorials that deploy notebook-based workflows for training models, packaging them for deployment, and testing those packages. The AI SDK assumes a machine-learning workflow that includes the following steps: - Training data preparation - Training models - Packaging models as an inference pipeline - Testing of packaged inference pipelines - Generating the inference pipeline for AI@Edge The AI SDK can be used both exploratively from interactive Python notebooks and purely programmatically as part of an automated ML workflow. [API documentation](docs/overview.html) # APIs for AI Software Development Kit (AI SDK) | API | | ---------------------------------------- | | [log_module](log_module/log_module.html) | | [simaticai](simaticai/__main__.html) | # log_module ### Logging in AI Inference Server The AI Inference Server's Python runtime provides a logger module which is embedded in the runtime and not available as a separate Python module. The AI SDK's log_module mimics the behavior of this embedded runtime logger by offering the same methods backed by the standard Python logging framework. This module is not a real logging framework, it only exists to help you write code against the logging framework on the Edge device. Using the AI SDK's LocalPipelineRunner the log_module wheel is automatically provided during the test run. If you want to run your code containing calls to log_module without the LocalPipelineRunner, you have to install the log_module wheel in your local execution environment. You should not install log_module on the Edge device itself. The log level can be set through the environment variable LOGLEVEL. Possible values are: TRACE, DEBUG, INFO, WARNING, ERROR, CRITICAL. Please note that this setting only applies for local testing. For running on the Edge device, you can set the log level in the AI Inference Server application. ## `LogModule` Facade class for exposing the AI Inference Server's log methods. This class can be imported and used the same way as on the Edge device. Example usage:: ```text from log_module import LogModule logger = LogModule() def run(data: str): logger.trace("trace from EMBEDDED Python") logger.info("info from EMBEDDED Python") logger.warning("warning from EMBEDDED Python") logger.warn("warn from EMBEDDED Python") logger.debug("debug from EMBEDDED Python") logger.error("error from EMBEDDED Python") logger.critical("critical from EMBEDDED Python") return {"ready": False, "output": None} ``` Source code in `docs/industrial-ai-suite/sdk/log_module/mock_logger.py` ```python class LogModule: """ Facade class for exposing the AI Inference Server's log methods. This class can be imported and used the same way as on the Edge device. Example usage:: from log_module import LogModule logger = LogModule() def run(data: str): logger.trace("trace from EMBEDDED Python") logger.info("info from EMBEDDED Python") logger.warning("warning from EMBEDDED Python") logger.warn("warn from EMBEDDED Python") logger.debug("debug from EMBEDDED Python") logger.error("error from EMBEDDED Python") logger.critical("critical from EMBEDDED Python") return {"ready": False, "output": None} """ def __init__(self): self.TRACE_LEVEL = 5 logging.addLevelName(self.TRACE_LEVEL, "TRACE") loglevel = os.environ.get("LOGLEVEL", "DEBUG").upper() if loglevel not in ["TRACE", "INFO", "WARNING", "ERROR", "CRITICAL"]: loglevel = "DEBUG" logging.basicConfig(level=loglevel) self.logger = logging.getLogger(__name__) self.logger.setLevel(loglevel) def trace(self, message): """ Logs a message on TRACE level. Args: message (str): The log message """ if self.logger.isEnabledFor(self.TRACE_LEVEL): self.logger.log(self.TRACE_LEVEL, _prefix(message)) def debug(self, message): """ Logs a message on DEBUG level. Args: message (str): The log message """ self.logger.debug(_prefix(message)) def info(self, message): """ Logs a message on INFO level. Args: message (str): The log message """ self.logger.info(_prefix(message)) def warning(self, message): """ Logs a message on WARNING level. Args: message (str): The log message """ self.logger.warning(_prefix(message)) def warn(self, message): """ Logs a message on WARNING level. Args: message (str): The log message """ self.warning(message) def error(self, message): """ Logs a message on ERROR level. Args: message (str): The log message """ self.logger.error(_prefix(message)) def critical(self, message): """ Logs a message on CRITICAL level. Args: message (str): The log message """ self.logger.critical(_prefix(message)) ``` ### `trace(message)` Logs a message on TRACE level. Parameters: | Name | Type | Description | Default | | --------- | ----- | --------------- | ---------- | | `message` | `str` | The log message | *required* | Source code in `docs/industrial-ai-suite/sdk/log_module/mock_logger.py` ```python def trace(self, message): """ Logs a message on TRACE level. Args: message (str): The log message """ if self.logger.isEnabledFor(self.TRACE_LEVEL): self.logger.log(self.TRACE_LEVEL, _prefix(message)) ``` ### `debug(message)` Logs a message on DEBUG level. Parameters: | Name | Type | Description | Default | | --------- | ----- | --------------- | ---------- | | `message` | `str` | The log message | *required* | Source code in `docs/industrial-ai-suite/sdk/log_module/mock_logger.py` ```python def debug(self, message): """ Logs a message on DEBUG level. Args: message (str): The log message """ self.logger.debug(_prefix(message)) ``` ### `info(message)` Logs a message on INFO level. Parameters: | Name | Type | Description | Default | | --------- | ----- | --------------- | ---------- | | `message` | `str` | The log message | *required* | Source code in `docs/industrial-ai-suite/sdk/log_module/mock_logger.py` ```python def info(self, message): """ Logs a message on INFO level. Args: message (str): The log message """ self.logger.info(_prefix(message)) ``` ### `warning(message)` Logs a message on WARNING level. Parameters: | Name | Type | Description | Default | | --------- | ----- | --------------- | ---------- | | `message` | `str` | The log message | *required* | Source code in `docs/industrial-ai-suite/sdk/log_module/mock_logger.py` ```python def warning(self, message): """ Logs a message on WARNING level. Args: message (str): The log message """ self.logger.warning(_prefix(message)) ``` ### `warn(message)` Logs a message on WARNING level. Parameters: | Name | Type | Description | Default | | --------- | ----- | --------------- | ---------- | | `message` | `str` | The log message | *required* | Source code in `docs/industrial-ai-suite/sdk/log_module/mock_logger.py` ```python def warn(self, message): """ Logs a message on WARNING level. Args: message (str): The log message """ self.warning(message) ``` ### `error(message)` Logs a message on ERROR level. Parameters: | Name | Type | Description | Default | | --------- | ----- | --------------- | ---------- | | `message` | `str` | The log message | *required* | Source code in `docs/industrial-ai-suite/sdk/log_module/mock_logger.py` ```python def error(self, message): """ Logs a message on ERROR level. Args: message (str): The log message """ self.logger.error(_prefix(message)) ``` ### `critical(message)` Logs a message on CRITICAL level. Parameters: | Name | Type | Description | Default | | --------- | ----- | --------------- | ---------- | | `message` | `str` | The log message | *required* | Source code in `docs/industrial-ai-suite/sdk/log_module/mock_logger.py` ```python def critical(self, message): """ Logs a message on CRITICAL level. Args: message (str): The log message """ self.logger.critical(_prefix(message)) ``` # mock_logger A logger facade for testing. This module can substitute the AI Inference Server's logger in development and testing environments. Each method prefixes the message with '[PY] ' and delegates to the standard Python logger. Limitation: The log methods can only accept a single string argument. As composition from multiple arguments is not supported on the Edge runtime, it is not supported in this module either. ## `LogModule` Facade class for exposing the AI Inference Server's log methods. This class can be imported and used the same way as on the Edge device. Example usage:: ```text from log_module import LogModule logger = LogModule() def run(data: str): logger.trace("trace from EMBEDDED Python") logger.info("info from EMBEDDED Python") logger.warning("warning from EMBEDDED Python") logger.warn("warn from EMBEDDED Python") logger.debug("debug from EMBEDDED Python") logger.error("error from EMBEDDED Python") logger.critical("critical from EMBEDDED Python") return {"ready": False, "output": None} ``` Source code in `docs/industrial-ai-suite/sdk/log_module/mock_logger.py` ```python class LogModule: """ Facade class for exposing the AI Inference Server's log methods. This class can be imported and used the same way as on the Edge device. Example usage:: from log_module import LogModule logger = LogModule() def run(data: str): logger.trace("trace from EMBEDDED Python") logger.info("info from EMBEDDED Python") logger.warning("warning from EMBEDDED Python") logger.warn("warn from EMBEDDED Python") logger.debug("debug from EMBEDDED Python") logger.error("error from EMBEDDED Python") logger.critical("critical from EMBEDDED Python") return {"ready": False, "output": None} """ def __init__(self): self.TRACE_LEVEL = 5 logging.addLevelName(self.TRACE_LEVEL, "TRACE") loglevel = os.environ.get("LOGLEVEL", "DEBUG").upper() if loglevel not in ["TRACE", "INFO", "WARNING", "ERROR", "CRITICAL"]: loglevel = "DEBUG" logging.basicConfig(level=loglevel) self.logger = logging.getLogger(__name__) self.logger.setLevel(loglevel) def trace(self, message): """ Logs a message on TRACE level. Args: message (str): The log message """ if self.logger.isEnabledFor(self.TRACE_LEVEL): self.logger.log(self.TRACE_LEVEL, _prefix(message)) def debug(self, message): """ Logs a message on DEBUG level. Args: message (str): The log message """ self.logger.debug(_prefix(message)) def info(self, message): """ Logs a message on INFO level. Args: message (str): The log message """ self.logger.info(_prefix(message)) def warning(self, message): """ Logs a message on WARNING level. Args: message (str): The log message """ self.logger.warning(_prefix(message)) def warn(self, message): """ Logs a message on WARNING level. Args: message (str): The log message """ self.warning(message) def error(self, message): """ Logs a message on ERROR level. Args: message (str): The log message """ self.logger.error(_prefix(message)) def critical(self, message): """ Logs a message on CRITICAL level. Args: message (str): The log message """ self.logger.critical(_prefix(message)) ``` ### `trace(message)` Logs a message on TRACE level. Parameters: | Name | Type | Description | Default | | --------- | ----- | --------------- | ---------- | | `message` | `str` | The log message | *required* | Source code in `docs/industrial-ai-suite/sdk/log_module/mock_logger.py` ```python def trace(self, message): """ Logs a message on TRACE level. Args: message (str): The log message """ if self.logger.isEnabledFor(self.TRACE_LEVEL): self.logger.log(self.TRACE_LEVEL, _prefix(message)) ``` ### `debug(message)` Logs a message on DEBUG level. Parameters: | Name | Type | Description | Default | | --------- | ----- | --------------- | ---------- | | `message` | `str` | The log message | *required* | Source code in `docs/industrial-ai-suite/sdk/log_module/mock_logger.py` ```python def debug(self, message): """ Logs a message on DEBUG level. Args: message (str): The log message """ self.logger.debug(_prefix(message)) ``` ### `info(message)` Logs a message on INFO level. Parameters: | Name | Type | Description | Default | | --------- | ----- | --------------- | ---------- | | `message` | `str` | The log message | *required* | Source code in `docs/industrial-ai-suite/sdk/log_module/mock_logger.py` ```python def info(self, message): """ Logs a message on INFO level. Args: message (str): The log message """ self.logger.info(_prefix(message)) ``` ### `warning(message)` Logs a message on WARNING level. Parameters: | Name | Type | Description | Default | | --------- | ----- | --------------- | ---------- | | `message` | `str` | The log message | *required* | Source code in `docs/industrial-ai-suite/sdk/log_module/mock_logger.py` ```python def warning(self, message): """ Logs a message on WARNING level. Args: message (str): The log message """ self.logger.warning(_prefix(message)) ``` ### `warn(message)` Logs a message on WARNING level. Parameters: | Name | Type | Description | Default | | --------- | ----- | --------------- | ---------- | | `message` | `str` | The log message | *required* | Source code in `docs/industrial-ai-suite/sdk/log_module/mock_logger.py` ```python def warn(self, message): """ Logs a message on WARNING level. Args: message (str): The log message """ self.warning(message) ``` ### `error(message)` Logs a message on ERROR level. Parameters: | Name | Type | Description | Default | | --------- | ----- | --------------- | ---------- | | `message` | `str` | The log message | *required* | Source code in `docs/industrial-ai-suite/sdk/log_module/mock_logger.py` ```python def error(self, message): """ Logs a message on ERROR level. Args: message (str): The log message """ self.logger.error(_prefix(message)) ``` ### `critical(message)` Logs a message on CRITICAL level. Parameters: | Name | Type | Description | Default | | --------- | ----- | --------------- | ---------- | | `message` | `str` | The log message | *required* | Source code in `docs/industrial-ai-suite/sdk/log_module/mock_logger.py` ```python def critical(self, message): """ Logs a message on CRITICAL level. Args: message (str): The log message """ self.logger.critical(_prefix(message)) ``` # AI Software Development Kit CLI ### python -m simaticai ```text usage: python -m simaticai [-h] {convert_package,create_delta_package,run_pipeline,telemetry} ... AI SDK command line interface. positional arguments: {convert_package,create_delta_package,run_pipeline,telemetry} convert_package Convert a Pipeline Configuration Package to an Edge Configuration Package that can be directly deployed. create_delta_package Create a Delta Configuration Package that can be deployed onto the original pipeline on AI Inference Server. run_pipeline Run an edge pipeline package locally. telemetry Manage your consent for telemetry data collection. options: -h, --help show this help message and exit ``` ### python -m simaticai convert_package ```text usage: python -m simaticai convert_package [-h] package_zip positional arguments: package_zip Path to the input package file. For "{path}/{name}_{version}.zip", the output file will be created as "{path}/{name}-edge_{version}.zip". If a file with such a name already exists, it is overwritten. options: -h, --help show this help message and exit ``` ### python -m simaticai create_delta_package ```text usage: python -m simaticai create_delta_package [-h] origin_package new_package positional arguments: origin_package Path to Origin Edge Package file. new_package Path to New Edge Package file. options: -h, --help show this help message and exit ``` ### python -m simaticai run_pipeline ```text usage: python -m simaticai run_pipeline [-h] [-c CONFIG_JSON] [--data DATA] [--test_dir TEST_DIR] package positional arguments: package Path to an Edge Package file. options: -h, --help show this help message and exit -c CONFIG_JSON, --config_json CONFIG_JSON A JSON configuration file that contains additional configuration for the data source. --data DATA Input data source --test_dir TEST_DIR Directory for the test environment ``` ### python -m simaticai telemetry ```text usage: python -m simaticai telemetry [--help] [--status] [--allow] [--deny] options: --help, -h Show information about telemetry data collection. --status, -s Show your consent statement about telemetry data collection. --allow Allow telemetry data collection. --deny Deny telemetry data collection. ``` # common Resource management module for SimaticAI SDK. This module provides functionality to manage and copy built-in SimaticAI SDK resources (such as ImageSet utilities) to user component directories. It handles source code copying, dependency management, and automatic requirements.txt updates. The module maintains a registry of available resources with their file paths and dependencies, allowing users to easily integrate common SDK utilities into their pipeline components. **Key Features:** - Resource registry management - Automatic dependency resolution - Resource file copying to target directories - Automatic update of requirements.txt file - Built-in support for ImageSet resource **Available Resources:** - ImageSet: Provides image dataset handling utilities with OpenCV dependencies Example Usage: ```python from simaticai.common.resources import copy_resource_to copy_resource_to("ImageSet", "my_component_source_dir") # This copies imageset.py and updates requirements.txt with necessary dependencies ``` **Functions:** - get_resource: Retrieves information about a specific SDK resource - copy_resource_to: Copies a resource file to a component directory with dependencies **Constants:** - SIMATICAI_RESOURCE_KEYS: List of available resource names - SIMATICAI_RESOURCES: Registry dictionary of all built-in resources ## `resources` ### `get_resource(resource_name)` Checks the availability of a SimaticAI SDK resource and retrieves information. Parameters: | Name | Type | Description | Default | | --------------- | ----- | --------------------------------------------------- | ---------- | | `resource_name` | `str` | Name of the SimaticAI SDK resource to be retrieved. | *required* | Source code in `docs/industrial-ai-suite/sdk/simaticai/common/resources.py` ```python def get_resource(resource_name: str): """ Checks the availability of a SimaticAI SDK resource and retrieves information. Args: resource_name (str): Name of the SimaticAI SDK resource to be retrieved. Returns: dict: Information about the resource including 'path', 'dependencies', and 'source' file location. Raises: AssertionError: If the resource_name is not supported. """ if resource_name not in SIMATICAI_RESOURCES: raise AssertionError(f"SimaticAI SDK resource '{resource_name}' is not supported.") resource_info = SIMATICAI_RESOURCES[resource_name] return { 'path': resource_info['path'], 'name': Path(resource_info['path']).name, 'dependencies': resource_info['dependencies'], 'source': module_resources.files("simaticai") / resource_info['path'] } ``` ### `copy_resource_to(resource_name, component_directory)` Copies a SimaticAI SDK resource file to the component directory. This method allows adding common SimaticAI SDK resource files such as `imageset.py` to a component. The method automatically adds the required dependencies for the resource file too. Parameters: | Name | Type | Description | Default | | --------------------- | ------------- | ------------------------------------------------------------------------------------- | ---------- | | `resource_name` | `str` | Name of the SimaticAI SDK resource to be copied. Currently supported only: 'ImageSet' | *required* | | `component_directory` | `path - like` | Root folder of your component's source code. | *required* | Source code in `docs/industrial-ai-suite/sdk/simaticai/common/resources.py` ```python def copy_resource_to(resource_name: str, component_directory: os.PathLike | str): """ Copies a SimaticAI SDK resource file to the component directory. This method allows adding common SimaticAI SDK resource files such as `imageset.py` to a component. The method automatically adds the required dependencies for the resource file too. Args: resource_name (str): Name of the SimaticAI SDK resource to be copied. Currently supported only: 'ImageSet' component_directory (path-like): Root folder of your component's source code. Raises: AssertionError: If the resource_name is not supported. """ resource = get_resource(resource_name) target_path = Path(component_directory) / resource['name'] if target_path.exists(): _logger.warning(f"Resource '{resource['name']}' is already added to target directory '{target_path}'.") else: _logger.warning(f"Adding SimaticAI SDK resource '{resource['name']}' to component directory '{component_directory}'. Creating file at '{target_path}'.") target_path.parent.mkdir(parents=True, exist_ok=True) shutil.copy(resource['source'], target_path) resource_dependencies = resource['dependencies'] not_required = set() requirements_path = Path(component_directory, 'requirements.txt') if requirements_path.is_file(): requirements = requirements_path.read_text(encoding='utf8').splitlines() requirements = [line.strip() for line in requirements if not line.strip().startswith('#')] for dependency in resource_dependencies: spec = pep508.parse_line(f"{dependency}") if any(spec.name in line for line in requirements): not_required.add(dependency) if len(not_required) < len(resource_dependencies): with requirements_path.open("a+", encoding="utf8") as req_file: req_file.write(f"\n# Additional dependencies required for '{resource_name}':") for dependency in set(resource_dependencies) - not_required: req_file.write(f"\n{dependency}") _logger.warning(f"Dependencies were added to {requirements_path}.") _logger.warning('Please make sure that every dependency version is compatible with your existing dependencies!') ``` # data This module contains additional files for AI SDK. These files are necessary for some functionalities of the SDK. ## `schemas` This module contains the schema files used in validation methods. These JSON schema files describe the configuration YAML files' schema for AI Inference Server. # deploy Pipeline packaging. This module contains classes and functionality for creating and validating pipeline configuration packages. ## `Component` Base class for pipeline components, with name, description, and a list of inputs and outputs. A new component is created with the given name and an empty input and output list. Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/component.py` ````python class Component: """Base class for pipeline components, with name, description, and a list of inputs and outputs. A new component is created with the given name and an empty input and output list. """ reserved_names = ["timestamp"] @dataclass class BatchInfo: """ Batch information for the component. This attribute specifies whether the component can handle batch input or output data. When set to True, the component will receive data in the form of a list of dictionaries instead of a single dictionary. It is important to note that the input and output variables on the component should still be defined as if they are single variables. If the input of the pipeline is configured for batch processing, it is recommended not to configure timeshifting, as the list will have the same timestamp for all elements, potentially resulting in data loss. """ inputBatch: bool = False outputBatch: bool = False def dict(self): return { 'inputBatch': 'Yes' if self.inputBatch is True else 'No', 'outputBatch': 'Yes' if self.outputBatch is True else 'No' } def __init__(self, name: str, desc: str = ""): """ Creates a new component with the given name and an empty input and output list. Args: name (str): Name of the component. desc (str): Optional description of the component """ if _allowed_characters_in_names.match(name) is None: raise AssertionError("Component name contains invalid character. The allowed characters are [-a-zA-Z0-9_].") self.name = name self.desc = desc self.inputs = {} self.outputs = {} self.batch = self.BatchInfo(False, False) def __repr__(self) -> str: text = f"[{self.__class__.__name__}] {self.name}\n" if self.desc != "": text += f"{self.desc}\n" if len(self.inputs) > 0: text += "\nComponent Inputs:\n" for name, input in self.inputs.items(): text += f"> {name} ({input['type']}){': ' + input['desc'] if input.get('desc') is not None else ''}\n" if len(self.outputs) > 0: text += "\nComponent Outputs:\n" for name, output in self.outputs.items(): text += f"< {name} ({output['type']}){': ' + output['desc'] if output.get('desc') is not None else ''}\n" return text def add_input(self, name: str, _type: str, desc: Optional[str] = None): """ Adds a new input to the component with its type. Name of the variables cannot be reserved name like 'timestamp'. Input variable 'timestamp' is a prebuilt key in the payload and its value contains the timestamp when the payload is created by AI Inference Server. Types supported by AI Inference Server version 1.6 are contained in the `type_dictionary`. Newer AI Inference server version may support additional types. In case the type is not known by the AI SDK, a warning message will be printed. The most frequently used types are - String: Typically used for data received from Databus - Object: Object type variables are designed to receive from Vision Connect or transfer images between components - Numeric scalar types: Typically used for data received from S7 Connector The example payload below shows the format of image received from VCA Connector ```python payload = { "image": { "resolutionWidth": image.width, "resolutionHeight": image.height, "mimeType": ["image/raw"], "dataType": "uint8", "channelsPerPixel": 3, "image": _swap_bytes(image.tobytes()) } } ``` Between components the format is the same format as the format of Object as an output. ```python "processedImage": { "metadata": json.dumps( { "resolutionWidth": image.width, "resolutionHeight": image.height } ), "bytes": image.tobytes() } ``` Args: name (str): Name of the new input. _type (str): Type of the new input. desc (str): Description of the input. (optional) """ if self.inputs is None: self.inputs = {} if name in self.inputs: raise AssertionError(f"Input '{name}' already exists.") if name.lower() in self.reserved_names: raise AssertionError(f"Input '{name}' is a reserved keyword.") flags = type_validation_flags(_type) if flags: flags_str = ", ".join([f.value for f in flags]) _logger.warning(f"WARNING! Type `{_type}` for input variable `{name}` has the following flags: {flags_str}. Please check if the target Inference Server supports this type.") self.inputs[name] = { "type": _type, } if desc is not None: self.inputs[name]['desc'] = desc def change_input(self, name: str, _type: str, desc: Optional[str] = None): """ Changes one of the inputs of the component. Args: name (str): Name of the input to be changed. _type (str): New type of the input. desc (str): Description of the input. (optional) """ if name not in self.inputs: raise AssertionError(f"There is no input with name '{name}'") flags = type_validation_flags(_type) if flags: flags_str = ", ".join([f.value for f in flags]) _logger.warning(f"WARNING! Type `{_type}` for input variable `{name}` has the following flags: {flags_str}. Please check if the target Inference Server supports this type.") self.inputs[name]['type'] = _type if desc is not None: self.inputs[name]['desc'] = desc def delete_input(self, name: str): """ Deletes an input from the component by name. Once the package has been created with the given component, it is recommended not to change the component directly. Instead, all necessary methods to change it are available through the package to avoid component inconsistencies. It is recommended to use `package.delete_input_wire(...)` with default parameter `with_input=True`. Args: name (str): Name of the input to be deleted. """ if name not in self.inputs: raise AssertionError(f"Component '{self.name}' has no input '{name}'") self.inputs.pop(name) def add_output(self, name: str, _type: str, desc: Optional[str] = None): """ Adds a new output to the component. Types supported by AI Inference Server version 1.6 are contained in the `type_dictionary`. Newer AI Inference server version may support additional types. In case the type is not known by the AI SDK, a warning message will be printed. The most frequently used types are - String: Typically used for data to be sent to Databus - Object: Typically used for images to be sent to ZMQ Connector - Numeric scalar types: Typically used for data sent to S7 Connector For outputs of type `Object` the entrypoint must return with a `dictionary` containing two fields, where one field has type `str` and the other field has type `bytes`. The example below shows the required format, assuming that 'image' is a PIL Image. ```python "processedImage": { "metadata": json.dumps( { "resolutionWidth": image.width, "resolutionHeight": image.height } ), "bytes": image.tobytes() } ``` Args: name (str): Name of the new output. _type (str): Type of the new output. desc (str): Description of the output. (optional) """ if self.outputs is None: self.outputs = {} if name in self.outputs: raise AssertionError(f"Output '{name}' already exists") if name.lower() in self.reserved_names: raise AssertionError(f"Output '{name}' is a reserved keyword.") flags = type_validation_flags(_type) if flags: flags_str = ", ".join([f.value for f in flags]) _logger.warning(f"WARNING! Type `{_type}` for output variable `{name}` has the following flags: {flags_str}. Please check if the target Inference Server supports this type.") self.outputs[name] = { "type": _type, } if desc is not None: self.outputs[name]['desc'] = desc def change_output(self, name: str, _type: str, desc: Optional[str] = None): """ Changes one of the outputs of the component. Args: name (str): Name of the output to be changed. _type (str): The new type of the output. desc (str): Description of the output. (optional) """ if name not in self.outputs: raise AssertionError(f"There is no output with name '{name}'") flags = type_validation_flags(_type) if flags: flags_str = ", ".join([f.value for f in flags]) _logger.warning(f"WARNING! Type `{_type}` for output variable `{name}` has the following flags: {flags_str}. Please check if the target Inference Server supports this type.") self.outputs[name]['type'] = _type if desc is not None: self.outputs[name]['desc'] = desc def delete_output(self, name: str): """ Deletes an output from the component by name. Once the package has been created with the given component, it is recommended not to change the component directly. Instead, all necessary methods to change it are available through the package to avoid component inconsistencies. Deleting an output which is represented in any wire will cause package inconsistency. Args: name (str): Name of the output to be deleted. """ if name not in self.outputs: raise AssertionError(f"Component '{self.name}' has no output '{name}'") self.outputs.pop(name) def _to_dict(self): inputs = [] inputs += [{ 'name': name, 'type': self.inputs[name]['type'], } for name in self.inputs] outputs = [] outputs += [{ 'name': name, 'type': self.outputs[name]['type'], 'metric': False, } for name in self.outputs] return { 'name': self.name, 'description': self.desc, 'batch': self.batch.dict(), 'inputType': inputs, 'outputType': outputs, } def validate(self): """ Empty method for child classess to implement. """ pass def save(self, destination, validate): """ Empty method for child classess to implement. """ pass ```` ### `BatchInfo` Batch information for the component. This attribute specifies whether the component can handle batch input or output data. When set to True, the component will receive data in the form of a list of dictionaries instead of a single dictionary. It is important to note that the input and output variables on the component should still be defined as if they are single variables. If the input of the pipeline is configured for batch processing, it is recommended not to configure timeshifting, as the list will have the same timestamp for all elements, potentially resulting in data loss. Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/component.py` ```python @dataclass class BatchInfo: """ Batch information for the component. This attribute specifies whether the component can handle batch input or output data. When set to True, the component will receive data in the form of a list of dictionaries instead of a single dictionary. It is important to note that the input and output variables on the component should still be defined as if they are single variables. If the input of the pipeline is configured for batch processing, it is recommended not to configure timeshifting, as the list will have the same timestamp for all elements, potentially resulting in data loss. """ inputBatch: bool = False outputBatch: bool = False def dict(self): return { 'inputBatch': 'Yes' if self.inputBatch is True else 'No', 'outputBatch': 'Yes' if self.outputBatch is True else 'No' } ``` ### `__init__(name, desc='')` Creates a new component with the given name and an empty input and output list. Parameters: | Name | Type | Description | Default | | ------ | ----- | ------------------------------------- | ---------- | | `name` | `str` | Name of the component. | *required* | | `desc` | `str` | Optional description of the component | `''` | Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/component.py` ```python def __init__(self, name: str, desc: str = ""): """ Creates a new component with the given name and an empty input and output list. Args: name (str): Name of the component. desc (str): Optional description of the component """ if _allowed_characters_in_names.match(name) is None: raise AssertionError("Component name contains invalid character. The allowed characters are [-a-zA-Z0-9_].") self.name = name self.desc = desc self.inputs = {} self.outputs = {} self.batch = self.BatchInfo(False, False) ``` ### `add_input(name, _type, desc=None)` Adds a new input to the component with its type. Name of the variables cannot be reserved name like 'timestamp'. Input variable 'timestamp' is a prebuilt key in the payload and its value contains the timestamp when the payload is created by AI Inference Server. Types supported by AI Inference Server version 1.6 are contained in the `type_dictionary`. Newer AI Inference server version may support additional types. In case the type is not known by the AI SDK, a warning message will be printed. The most frequently used types are - String: Typically used for data received from Databus - Object: Object type variables are designed to receive from Vision Connect or transfer images between components - Numeric scalar types: Typically used for data received from S7 Connector The example payload below shows the format of image received from VCA Connector ```python payload = { "image": { "resolutionWidth": image.width, "resolutionHeight": image.height, "mimeType": ["image/raw"], "dataType": "uint8", "channelsPerPixel": 3, "image": _swap_bytes(image.tobytes()) } } ``` Between components the format is the same format as the format of Object as an output. ```python "processedImage": { "metadata": json.dumps( { "resolutionWidth": image.width, "resolutionHeight": image.height } ), "bytes": image.tobytes() } ``` Parameters: | Name | Type | Description | Default | | ------- | ----- | ------------------------------------ | ---------- | | `name` | `str` | Name of the new input. | *required* | | `_type` | `str` | Type of the new input. | *required* | | `desc` | `str` | Description of the input. (optional) | `None` | Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/component.py` ````python def add_input(self, name: str, _type: str, desc: Optional[str] = None): """ Adds a new input to the component with its type. Name of the variables cannot be reserved name like 'timestamp'. Input variable 'timestamp' is a prebuilt key in the payload and its value contains the timestamp when the payload is created by AI Inference Server. Types supported by AI Inference Server version 1.6 are contained in the `type_dictionary`. Newer AI Inference server version may support additional types. In case the type is not known by the AI SDK, a warning message will be printed. The most frequently used types are - String: Typically used for data received from Databus - Object: Object type variables are designed to receive from Vision Connect or transfer images between components - Numeric scalar types: Typically used for data received from S7 Connector The example payload below shows the format of image received from VCA Connector ```python payload = { "image": { "resolutionWidth": image.width, "resolutionHeight": image.height, "mimeType": ["image/raw"], "dataType": "uint8", "channelsPerPixel": 3, "image": _swap_bytes(image.tobytes()) } } ``` Between components the format is the same format as the format of Object as an output. ```python "processedImage": { "metadata": json.dumps( { "resolutionWidth": image.width, "resolutionHeight": image.height } ), "bytes": image.tobytes() } ``` Args: name (str): Name of the new input. _type (str): Type of the new input. desc (str): Description of the input. (optional) """ if self.inputs is None: self.inputs = {} if name in self.inputs: raise AssertionError(f"Input '{name}' already exists.") if name.lower() in self.reserved_names: raise AssertionError(f"Input '{name}' is a reserved keyword.") flags = type_validation_flags(_type) if flags: flags_str = ", ".join([f.value for f in flags]) _logger.warning(f"WARNING! Type `{_type}` for input variable `{name}` has the following flags: {flags_str}. Please check if the target Inference Server supports this type.") self.inputs[name] = { "type": _type, } if desc is not None: self.inputs[name]['desc'] = desc ```` ### `change_input(name, _type, desc=None)` Changes one of the inputs of the component. Parameters: | Name | Type | Description | Default | | ------- | ----- | ------------------------------------ | ---------- | | `name` | `str` | Name of the input to be changed. | *required* | | `_type` | `str` | New type of the input. | *required* | | `desc` | `str` | Description of the input. (optional) | `None` | Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/component.py` ```python def change_input(self, name: str, _type: str, desc: Optional[str] = None): """ Changes one of the inputs of the component. Args: name (str): Name of the input to be changed. _type (str): New type of the input. desc (str): Description of the input. (optional) """ if name not in self.inputs: raise AssertionError(f"There is no input with name '{name}'") flags = type_validation_flags(_type) if flags: flags_str = ", ".join([f.value for f in flags]) _logger.warning(f"WARNING! Type `{_type}` for input variable `{name}` has the following flags: {flags_str}. Please check if the target Inference Server supports this type.") self.inputs[name]['type'] = _type if desc is not None: self.inputs[name]['desc'] = desc ``` ### `delete_input(name)` Deletes an input from the component by name. Once the package has been created with the given component, it is recommended not to change the component directly. Instead, all necessary methods to change it are available through the package to avoid component inconsistencies. It is recommended to use `package.delete_input_wire(...)` with default parameter `with_input=True`. Parameters: | Name | Type | Description | Default | | ------ | ----- | -------------------------------- | ---------- | | `name` | `str` | Name of the input to be deleted. | *required* | Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/component.py` ```python def delete_input(self, name: str): """ Deletes an input from the component by name. Once the package has been created with the given component, it is recommended not to change the component directly. Instead, all necessary methods to change it are available through the package to avoid component inconsistencies. It is recommended to use `package.delete_input_wire(...)` with default parameter `with_input=True`. Args: name (str): Name of the input to be deleted. """ if name not in self.inputs: raise AssertionError(f"Component '{self.name}' has no input '{name}'") self.inputs.pop(name) ``` ### `add_output(name, _type, desc=None)` Adds a new output to the component. Types supported by AI Inference Server version 1.6 are contained in the `type_dictionary`. Newer AI Inference server version may support additional types. In case the type is not known by the AI SDK, a warning message will be printed. The most frequently used types are - String: Typically used for data to be sent to Databus - Object: Typically used for images to be sent to ZMQ Connector - Numeric scalar types: Typically used for data sent to S7 Connector For outputs of type `Object` the entrypoint must return with a `dictionary` containing two fields, where one field has type `str` and the other field has type `bytes`. The example below shows the required format, assuming that 'image' is a PIL Image. ```python "processedImage": { "metadata": json.dumps( { "resolutionWidth": image.width, "resolutionHeight": image.height } ), "bytes": image.tobytes() } ``` Parameters: | Name | Type | Description | Default | | ------- | ----- | ------------------------------------- | ---------- | | `name` | `str` | Name of the new output. | *required* | | `_type` | `str` | Type of the new output. | *required* | | `desc` | `str` | Description of the output. (optional) | `None` | Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/component.py` ````python def add_output(self, name: str, _type: str, desc: Optional[str] = None): """ Adds a new output to the component. Types supported by AI Inference Server version 1.6 are contained in the `type_dictionary`. Newer AI Inference server version may support additional types. In case the type is not known by the AI SDK, a warning message will be printed. The most frequently used types are - String: Typically used for data to be sent to Databus - Object: Typically used for images to be sent to ZMQ Connector - Numeric scalar types: Typically used for data sent to S7 Connector For outputs of type `Object` the entrypoint must return with a `dictionary` containing two fields, where one field has type `str` and the other field has type `bytes`. The example below shows the required format, assuming that 'image' is a PIL Image. ```python "processedImage": { "metadata": json.dumps( { "resolutionWidth": image.width, "resolutionHeight": image.height } ), "bytes": image.tobytes() } ``` Args: name (str): Name of the new output. _type (str): Type of the new output. desc (str): Description of the output. (optional) """ if self.outputs is None: self.outputs = {} if name in self.outputs: raise AssertionError(f"Output '{name}' already exists") if name.lower() in self.reserved_names: raise AssertionError(f"Output '{name}' is a reserved keyword.") flags = type_validation_flags(_type) if flags: flags_str = ", ".join([f.value for f in flags]) _logger.warning(f"WARNING! Type `{_type}` for output variable `{name}` has the following flags: {flags_str}. Please check if the target Inference Server supports this type.") self.outputs[name] = { "type": _type, } if desc is not None: self.outputs[name]['desc'] = desc ```` ### `change_output(name, _type, desc=None)` Changes one of the outputs of the component. Parameters: | Name | Type | Description | Default | | ------- | ----- | ------------------------------------- | ---------- | | `name` | `str` | Name of the output to be changed. | *required* | | `_type` | `str` | The new type of the output. | *required* | | `desc` | `str` | Description of the output. (optional) | `None` | Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/component.py` ```python def change_output(self, name: str, _type: str, desc: Optional[str] = None): """ Changes one of the outputs of the component. Args: name (str): Name of the output to be changed. _type (str): The new type of the output. desc (str): Description of the output. (optional) """ if name not in self.outputs: raise AssertionError(f"There is no output with name '{name}'") flags = type_validation_flags(_type) if flags: flags_str = ", ".join([f.value for f in flags]) _logger.warning(f"WARNING! Type `{_type}` for output variable `{name}` has the following flags: {flags_str}. Please check if the target Inference Server supports this type.") self.outputs[name]['type'] = _type if desc is not None: self.outputs[name]['desc'] = desc ``` ### `delete_output(name)` Deletes an output from the component by name. Once the package has been created with the given component, it is recommended not to change the component directly. Instead, all necessary methods to change it are available through the package to avoid component inconsistencies. Deleting an output which is represented in any wire will cause package inconsistency. Parameters: | Name | Type | Description | Default | | ------ | ----- | --------------------------------- | ---------- | | `name` | `str` | Name of the output to be deleted. | *required* | Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/component.py` ```python def delete_output(self, name: str): """ Deletes an output from the component by name. Once the package has been created with the given component, it is recommended not to change the component directly. Instead, all necessary methods to change it are available through the package to avoid component inconsistencies. Deleting an output which is represented in any wire will cause package inconsistency. Args: name (str): Name of the output to be deleted. """ if name not in self.outputs: raise AssertionError(f"Component '{self.name}' has no output '{name}'") self.outputs.pop(name) ``` ### `validate()` Empty method for child classess to implement. Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/component.py` ```python def validate(self): """ Empty method for child classess to implement. """ pass ``` ### `save(destination, validate)` Empty method for child classess to implement. Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/component.py` ```python def save(self, destination, validate): """ Empty method for child classess to implement. """ pass ``` ## `GPURuntimeComponent` Bases: `Component` The GPURuntimeComponent is used to define a component that runs on a GPU device. The component works only with ONNX models and can be used in an Inference Pipeline. Attributes: | Name | Type | Description | | --------- | ----- | ---------------------- | | `name` | `str` | Component name. | | `version` | `str` | Component version. | | `desc` | `str` | Component description. | Methods: | Name | Description | | ------------ | ----------- | | `use_model` | Path | | `use_config` | Path | | `save` | Path | Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/gpuruntime_component.py` ```python class GPURuntimeComponent(Component): """ The GPURuntimeComponent is used to define a component that runs on a GPU device. The component works only with ONNX models and can be used in an Inference Pipeline. Attributes: name (str): Component name. version (str): Component version. desc (str): Component description. Methods: use_model(self, path: Path | str, max_batch_size: int, optimization: Optional[model_config.TensorRTOptimization] = None, warmup: model_config.Warmup = None): Add an ONNX model file for the component. use_config(self, path: Path | str): Use a custom config.pbtxt file instead of the autogenerated one. save(self, destination: Path | str, validate = False): Saves the component to a folder structure, so it can be used as part of a pipeline configuration package. """ def __init__(self, name: str = "inference", version: str = "1", desc: str = ""): """ Creates a new, empty GPU Runtime component. Args: name (str): Component name. (default: inference) version (str): Component version. (default: 1) desc (str): Component description (optional) """ super().__init__(name=name, desc=desc) self.version: str = version self.entrypoint: Path | None = None self.model_path: Path | None = None self.model_version: str = "1" self.config: Path | None = None self.auto_config = None self.warmup: bool = False self.optimization: bool = False def _to_dict(self): return { **super()._to_dict(), 'version': self.version, 'entrypoint': f"{self.model_version}/{self.entrypoint.name}", 'hwType': 'GPU', 'warmup': self.warmup, 'optimization': self.optimization, 'runtime': { 'type': 'gpuruntime', 'version': '0.1.0', } } def use_model(self, path: Path | str, max_batch_size: int, optimization: model_config.TensorRTOptimization | None = None, warmup: model_config.Warmup | bool | None = None): """ Add the ONNX model file for the component. Args: path (Path | str): The path to the ONNX model file. max_batch_size (int): The maximum batch size for the model. optimization (model_config.TensorRTOptimization, optional): The optimization configuration for the model. Defaults to None. warmup (model_config.Warmup, optional): The warmup configuration for the model. Defaults to None. Raises: AssertionError: If the specified model file is not found, has an invalid extension, or if max_batch_size is less than 0. """ path = Path(path) if not path.is_file(): raise AssertionError(f"specified model file not found: '{path}'") if path.suffix != ".onnx": raise AssertionError(f"model file extension is not '.onnx': '{path}'") if max_batch_size < 0: raise AssertionError("max_batch_size must be greater or equal to 0") self.entrypoint = Path("model.onnx") self.model_path = path if self.config is not None: _logger.warning("Previously added configuration was removed. Component will use the default configuration unless you specify your own.") self.config = None # Removing old automatic variables if self.auto_config is not None: for var in self.auto_config.inputs: self.delete_input(var["name"]) for var in self.auto_config.outputs: self.delete_output(var["name"]) model_warmup = warmup if isinstance(warmup, bool): self.warmup = warmup model_warmup = model_config.Warmup.ZERO_DATA if warmup else model_config.Warmup.DISABLED elif isinstance(warmup, model_config.Warmup): self.warmup = warmup != model_config.Warmup.DISABLED else: self.warmup = False model_warmup = model_config.Warmup.DISABLED self.optimization = isinstance(optimization, model_config.TensorRTOptimization) self.auto_config = model_config.ModelConfig(onnx_path=path, max_batch_size=max_batch_size, warmup=model_warmup, optimization=optimization) for var in self.auto_config.inputs: self.add_input(var["name"], var["type"]) for var in self.auto_config.outputs: self.add_output(var["name"], var["type"]) def use_config(self, path: Path | str): """ Sets the configuration file to be used for inference. Intended usage is to use a custom configuration file instead of the autogenerated one. This way extra configurations can be added to the component, such as the execution accelerator. Args: path (Path | str): The path to the configuration file. Raises: AssertionError: If the specified config file is not found or has an invalid extension. """ path = Path(path) if not path.is_file(): raise AssertionError(f"specified config file not found: '{path}'") if path.suffix != ".pbtxt": raise AssertionError(f"config file extension is not '.pbtxt': '{path}'") config_data = _validate_and_load_gpuruntime_config(path) self.warmup = (0 < len(config_data.model_warmup)) self.optimization = (0 < len(f"{config_data.optimization}")) self.config = path def save(self, destination: Path | str, validate = False): """ Saves the component to a folder structure, so it can be used as part of a pipeline configuration package. The component folder contains the following: - An `.onnx` model file - A `.pbtxt` configuration file Args: destination (path-like): Target directory to which the component will be saved. """ if self.entrypoint is None: raise AssertionError("An ONNX model file must be specified before the component can be saved.") component_dir = Path(destination) / self.name component_dir.mkdir(parents = True, exist_ok = True) model_dir = component_dir / self.model_version model_dir.mkdir(exist_ok = True) shutil.copy(str(self.model_path), str(model_dir / "model.onnx")) if self.config is None: _logger.warning("Configuration was not specified. Model will be saved with default configuration.") (component_dir / "config.pbtxt").write_text(f"{self.auto_config}") else: shutil.copy(self.config, component_dir) ``` ### `__init__(name='inference', version='1', desc='')` Creates a new, empty GPU Runtime component. Parameters: | Name | Type | Description | Default | | --------- | ----- | ------------------------------------ | ------------- | | `name` | `str` | Component name. (default: inference) | `'inference'` | | `version` | `str` | Component version. (default: 1) | `'1'` | | `desc` | `str` | Component description (optional) | `''` | Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/gpuruntime_component.py` ```python def __init__(self, name: str = "inference", version: str = "1", desc: str = ""): """ Creates a new, empty GPU Runtime component. Args: name (str): Component name. (default: inference) version (str): Component version. (default: 1) desc (str): Component description (optional) """ super().__init__(name=name, desc=desc) self.version: str = version self.entrypoint: Path | None = None self.model_path: Path | None = None self.model_version: str = "1" self.config: Path | None = None self.auto_config = None self.warmup: bool = False self.optimization: bool = False ``` ### `use_model(path, max_batch_size, optimization=None, warmup=None)` Add the ONNX model file for the component. Parameters: | Name | Type | Description | Default | | ---------------- | ---------------------- | --------------------------------------------------------------- | -------------------------------- | | `path` | \`Path | str\` | The path to the ONNX model file. | | `max_batch_size` | `int` | The maximum batch size for the model. | *required* | | `optimization` | `TensorRTOptimization` | The optimization configuration for the model. Defaults to None. | `None` | | `warmup` | `Warmup` | The warmup configuration for the model. Defaults to None. | `None` | Raises: | Type | Description | | ---------------- | -------------------------------------------------------------------------------------------------------- | | `AssertionError` | If the specified model file is not found, has an invalid extension, or if max_batch_size is less than 0. | Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/gpuruntime_component.py` ```python def use_model(self, path: Path | str, max_batch_size: int, optimization: model_config.TensorRTOptimization | None = None, warmup: model_config.Warmup | bool | None = None): """ Add the ONNX model file for the component. Args: path (Path | str): The path to the ONNX model file. max_batch_size (int): The maximum batch size for the model. optimization (model_config.TensorRTOptimization, optional): The optimization configuration for the model. Defaults to None. warmup (model_config.Warmup, optional): The warmup configuration for the model. Defaults to None. Raises: AssertionError: If the specified model file is not found, has an invalid extension, or if max_batch_size is less than 0. """ path = Path(path) if not path.is_file(): raise AssertionError(f"specified model file not found: '{path}'") if path.suffix != ".onnx": raise AssertionError(f"model file extension is not '.onnx': '{path}'") if max_batch_size < 0: raise AssertionError("max_batch_size must be greater or equal to 0") self.entrypoint = Path("model.onnx") self.model_path = path if self.config is not None: _logger.warning("Previously added configuration was removed. Component will use the default configuration unless you specify your own.") self.config = None # Removing old automatic variables if self.auto_config is not None: for var in self.auto_config.inputs: self.delete_input(var["name"]) for var in self.auto_config.outputs: self.delete_output(var["name"]) model_warmup = warmup if isinstance(warmup, bool): self.warmup = warmup model_warmup = model_config.Warmup.ZERO_DATA if warmup else model_config.Warmup.DISABLED elif isinstance(warmup, model_config.Warmup): self.warmup = warmup != model_config.Warmup.DISABLED else: self.warmup = False model_warmup = model_config.Warmup.DISABLED self.optimization = isinstance(optimization, model_config.TensorRTOptimization) self.auto_config = model_config.ModelConfig(onnx_path=path, max_batch_size=max_batch_size, warmup=model_warmup, optimization=optimization) for var in self.auto_config.inputs: self.add_input(var["name"], var["type"]) for var in self.auto_config.outputs: self.add_output(var["name"], var["type"]) ``` ### `use_config(path)` Sets the configuration file to be used for inference. Intended usage is to use a custom configuration file instead of the autogenerated one. This way extra configurations can be added to the component, such as the execution accelerator. Parameters: | Name | Type | Description | Default | | ------ | ------ | ----------- | ----------------------------------- | | `path` | \`Path | str\` | The path to the configuration file. | Raises: | Type | Description | | ---------------- | ---------------------------------------------------------------------- | | `AssertionError` | If the specified config file is not found or has an invalid extension. | Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/gpuruntime_component.py` ```python def use_config(self, path: Path | str): """ Sets the configuration file to be used for inference. Intended usage is to use a custom configuration file instead of the autogenerated one. This way extra configurations can be added to the component, such as the execution accelerator. Args: path (Path | str): The path to the configuration file. Raises: AssertionError: If the specified config file is not found or has an invalid extension. """ path = Path(path) if not path.is_file(): raise AssertionError(f"specified config file not found: '{path}'") if path.suffix != ".pbtxt": raise AssertionError(f"config file extension is not '.pbtxt': '{path}'") config_data = _validate_and_load_gpuruntime_config(path) self.warmup = (0 < len(config_data.model_warmup)) self.optimization = (0 < len(f"{config_data.optimization}")) self.config = path ``` ### `save(destination, validate=False)` Saves the component to a folder structure, so it can be used as part of a pipeline configuration package. The component folder contains the following: - An `.onnx` model file - A `.pbtxt` configuration file Parameters: | Name | Type | Description | Default | | ------------- | ------------- | ------------------------------------------------------ | ---------- | | `destination` | `path - like` | Target directory to which the component will be saved. | *required* | Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/gpuruntime_component.py` ```python def save(self, destination: Path | str, validate = False): """ Saves the component to a folder structure, so it can be used as part of a pipeline configuration package. The component folder contains the following: - An `.onnx` model file - A `.pbtxt` configuration file Args: destination (path-like): Target directory to which the component will be saved. """ if self.entrypoint is None: raise AssertionError("An ONNX model file must be specified before the component can be saved.") component_dir = Path(destination) / self.name component_dir.mkdir(parents = True, exist_ok = True) model_dir = component_dir / self.model_version model_dir.mkdir(exist_ok = True) shutil.copy(str(self.model_path), str(model_dir / "model.onnx")) if self.config is None: _logger.warning("Configuration was not specified. Model will be saved with default configuration.") (component_dir / "config.pbtxt").write_text(f"{self.auto_config}") else: shutil.copy(self.config, component_dir) ``` ## `PipelineData` Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/pipeline_data.py` ```python class PipelineData: def __init__(self, name: str, version: Optional[str] = None, desc: str = ""): """ A newly initialized `Pipeline` will contain no `Component` or wire, just its name and version will be set. The name and version will define together the name of the zip file when the package is saved. Args: name (str): Name of the package desc (str): Package description (optional) version (str): Version of the package """ self.name = name self.desc = desc self.version = version self.init_version = version # initial version; used when version is not set in Pipeline.export() and save() self.save_version = version # contains the version determined at exporting/saving self.package_id: Optional[uuid.UUID] = None self.author = 'AI SDK' self.components = {} self.wiring = {} self.parameters: list[PipelineParameter] = [] self.periodicity = None self.timeshift_reference = [] self.inputs: list[PipelineVariable] = [] self.outputs: list[PipelineVariable] = [] self.log_level = logging.INFO self.cycle_time = None # Helper method returns index or None if the pipeline inputs/outputs contain the given variable def _get_io_index(self, io_list: list[PipelineVariable], variable: str) -> Optional[int]: for i, io in enumerate(io_list): if io.variableName == variable: return i return None def _get_io_variable(self, variable_name: str) -> Optional[PipelineVariable]: for io in self.inputs + self.outputs: if io.variableName == variable_name: return io return None def _get_parameter(self, parameter_name: str) -> Optional[PipelineParameter]: for param in self.parameters: if param.name == parameter_name: return param return None def _get_parameter_index(self, parameter_name: str) -> Optional[int]: for i, param in enumerate(self.parameters): if param.name == parameter_name: return i return None ``` ### `__init__(name, version=None, desc='')` A newly initialized `Pipeline` will contain no `Component` or wire, just its name and version will be set. The name and version will define together the name of the zip file when the package is saved. Parameters: | Name | Type | Description | Default | | --------- | ----- | ------------------------------ | ---------- | | `name` | `str` | Name of the package | *required* | | `desc` | `str` | Package description (optional) | `''` | | `version` | `str` | Version of the package | `None` | Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/pipeline_data.py` ```python def __init__(self, name: str, version: Optional[str] = None, desc: str = ""): """ A newly initialized `Pipeline` will contain no `Component` or wire, just its name and version will be set. The name and version will define together the name of the zip file when the package is saved. Args: name (str): Name of the package desc (str): Package description (optional) version (str): Version of the package """ self.name = name self.desc = desc self.version = version self.init_version = version # initial version; used when version is not set in Pipeline.export() and save() self.save_version = version # contains the version determined at exporting/saving self.package_id: Optional[uuid.UUID] = None self.author = 'AI SDK' self.components = {} self.wiring = {} self.parameters: list[PipelineParameter] = [] self.periodicity = None self.timeshift_reference = [] self.inputs: list[PipelineVariable] = [] self.outputs: list[PipelineVariable] = [] self.log_level = logging.INFO self.cycle_time = None ``` ## `save_readme_html(pipeline, destination)` Saves a `README.html` in the `destination` folder that describes the pipeline. Parameters: | Name | Type | Description | Default | | ------------- | ------------- | ------------------------------- | ---------- | | `destination` | `path - like` | Path of the destination folder. | *required* | Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/pipeline_page.py` ```python def save_readme_html(pipeline: PipelineData, destination: Union[str, os.PathLike]): """ Saves a `README.html` in the `destination` folder that describes the pipeline. Args: destination (path-like): Path of the destination folder. """ pipelinePage = _PipelinePage(pipeline) readme_html_path = Path(destination) / README_HTML readme_html_path.write_text(pipelinePage.__str__()) ``` ## `Pipeline` Bases: `PipelineData` `Pipeline` represents a pipeline configuration package with `Components` and wires to provide a data flow on the AI Inference Server. The `Components` have inputs and outputs to transfer data to each other and the wires describe this data flow between them. The package also contains configuration files required to deploy a pipeline on an Industrial Edge device. A newly initialized `Pipeline` does not contain any `Component` or wire, only its name and version will be set. The name and version together will define the name of the zip file when the package is saved. Parameters: | Name | Type | Description | Default | | --------- | ----- | ---------------------- | ---------- | | `name` | `str` | Name of the package | *required* | | `version` | `str` | Version of the package | `None` | Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/pipeline.py` ```python class Pipeline(PipelineData): """ `Pipeline` represents a pipeline configuration package with `Components` and wires to provide a data flow on the AI Inference Server. The `Components` have inputs and outputs to transfer data to each other and the wires describe this data flow between them. The package also contains configuration files required to deploy a pipeline on an Industrial Edge device. A newly initialized `Pipeline` does not contain any `Component` or wire, only its name and version will be set. The name and version together will define the name of the zip file when the package is saved. Args: name (str): Name of the package version (str): Version of the package """ _wire_hash_string = "{}.{} -> {}.{}" def __init__(self, name: str, version: Optional[str] = None, desc: str = ""): """ A newly initialized `Pipeline` will contain no `Component` or wire, just its name and version will be set. The name and version will define together the name of the zip file when the package is saved. Args: name (str): Name of the package desc (str): Package description (optional) version (str): Version of the package """ super().__init__(name, version, desc) self.report_writer = PipelineReportWriter() report_writer_handler = ReportWriterHandler(self.report_writer) _logger.addHandler(report_writer_handler) _python_dependencies_logger.addHandler(report_writer_handler) _wheelhouse_logger.addHandler(report_writer_handler) def _set_log_level(self, log_level: int): self.log_level = log_level _logger.setLevel(self.log_level) @staticmethod def from_components(components: list, name: str, version: Optional[str] = None, desc: str = "") -> "Pipeline": """ Creates a pipeline configuration from the given components. The components are linked in a linear sequence with inputs and outputs auto-wired based on the name of the inputs and outputs of the components. The inputs of the first component will be wired as the pipeline inputs and the outputs of the last component will be wired as the pipeline outputs. The components must have unique names. Two or more versions of the same component can not be packaged simultaneously without renaming them. Args: components (list): List of PythonComponents name (str): Name of the pipeline version (str): Version information of the pipeline. (Optional) Returns: Pipeline: Pipeline object with the auto-wired components """ pipeline = Pipeline(name, version, desc=desc) first_component = components[0] pipeline.add_component(first_component) pipeline.inputs = [PipelineVariable(first_component.name, component_input) for component_input in first_component.inputs] pipeline.outputs = [PipelineVariable(first_component.name, output) for output in first_component.outputs] for component in components[1:]: pipeline.add_component(component) for t_output in pipeline.outputs: try: pipeline.add_wiring(t_output.componentName, t_output.variableName, component.name, t_output.variableName) except Exception as e: _logger.warning(f"Output variable {t_output.componentName}.{t_output.variableName} couldn't be auto-wired.\nCause: {e}") unwired_variables = [f'{component.name}.{x}' for x in component.inputs if not any(s.endswith(f'{component.name}.{x}') for s in pipeline.wiring)] if len(unwired_variables) > 0: for variable in unwired_variables: _logger.warning(f"Input variable {variable} couldn't be auto-wired.\n") pipeline.outputs = [PipelineVariable(component.name, output) for output in component.outputs] # ImageSet is not allowed in pipeline outputs for i, o in enumerate(pipeline.outputs): if pipeline.components[o.componentName].outputs[o.variableName]["type"].lower() == 'imageset': _logger.warning(f"Output variable {o.componentName}.{o.variableName} of type 'ImageSet' can not be added as pipeline output.") pipeline.outputs.pop(i) return pipeline def __repr__(self) -> str: """ Textual representation of the configured package. The method shows the `Components` with their inputs, outputs and parameters as well as the wiring between these `Components`. Returns: [str]: Textual representation of the package """ version = self.save_version if self.save_version is not None else self.init_version text = f"[{self.__class__.__name__}] {self.name} ({version})\n" if self.desc != "": text += f"{self.desc}\n" if len(self.parameters) > 0: text += "\nPipeline Parameters:\n" for parameter in self.parameters: text += f"- {parameter.name} ({parameter.dtype}, default: '{parameter.defaultValue}'){(': ' + parameter.description) if parameter.description is not None else ''}\n" if len(self.inputs) > 0: text += "\nPipeline Inputs:\n" for p_input in self.inputs: c_input = self.components[p_input.componentName].inputs[p_input.variableName] text += f"> {p_input.variableName} ({c_input['type']}){': ' + c_input['desc'] if c_input.get('desc') is not None else ''}\n" if len(self.outputs) > 0: text += "\nPipeline Outputs:\n" for p_output in self.outputs: c_output = self.components[p_output.componentName].outputs[p_output.variableName] text += f"< {p_output.variableName} ({c_output['type']}){': ' + c_output['desc'] if c_output.get('desc') is not None else ''}\n" metrics = [(name, metric, component_name) for component_name, component in self.components.items() if isinstance(component, PythonComponent) for name, metric in component.metrics.items()] if len(metrics) > 0: text += "\nMetrics:\n" for name, metric, _ in metrics: text += f"< {name}{': ' + metric['desc'] if metric.get('desc') is not None else ''}\n" if len(self.wiring) > 0: text += "\nI/O Wiring:\n" for p_input in self.inputs: text += f" {p_input.variableName} -> {p_input.componentName}.{p_input.variableName}\n" for wire_hash in self.wiring: text += f" {wire_hash}\n" for p_output in self.outputs: text += f" {p_output.componentName}.{p_output.variableName} -> {p_output.variableName}\n" for name, metric, component_name in metrics: text += f" {component_name}.{name} -> {name}\n" if self.periodicity is not None: text += "\nTimeshifting:\n" text += f" Periodicity: {self.periodicity} ms\n" if len(self.timeshift_reference) > 0: text += " References:\n" for ref in self.timeshift_reference: text += f" - {ref}\n" if self.cycle_time is not None: text += f"\nCycle Time: {self.cycle_time} ms\n" for component in self.components.values(): text += "\n" + component.__repr__() return text def add_input(self, component, variable): """ Defines an input variable on the given component as a pipeline input. Args: component (str): Name of the component variable (str): Name of the input variable """ try: _ = self.components[component].inputs[variable] except KeyError: raise AssertionError("The component with input variable must exist in the pipeline.") if self.inputs is None: self.inputs = [] if self._get_io_index(self.inputs, variable) is not None: raise AssertionError("The pipeline input already exists.") self.inputs.append(PipelineVariable(componentName=component, variableName=variable)) def delete_input(self, component: str, variable: str): """ Deletes a pipeline input. Args: component (str): Name of the component variable (str): Name of the input variable """ if (idx := self._get_io_index(self.inputs, variable)) is None: raise AssertionError("The pipeline input does not exist.") self.inputs.pop(idx) def add_output(self, component, variable): """ Defines an output variable on the given component as a pipeline output. Args: component (str): Name of the component variable (str): Name of the output variable """ try: output = self.components[component].outputs[variable] except KeyError: raise AssertionError("The component with output variable must exist in the pipeline.") if self.outputs is None: self.outputs = [] if self._get_io_index(self.outputs, variable) is not None: raise AssertionError("The pipeline output already exists.") if output["type"].lower() == 'imageset': _logger.warning("Output variables of type 'ImageSet' can not be added as pipeline outputs.") self.outputs.append(PipelineVariable(componentName=component, variableName=variable)) def delete_output(self, component: str, variable: str): """ Deletes a pipeline output. Args: component (str): Name of the component variable (str): Name of the output variable """ if (idx := self._get_io_index(self.outputs, variable)) is None: raise AssertionError("The pipeline output does not exist.") self.outputs.pop(idx) def add_component(self, component: Component): """ Adds a `Component` to the pipeline configuration without any connection. The component must have a unique name. Two or more versions of the same component can not be added to the same pipeline with the same component name. Args: component (Component): `Component` to be added """ if component.name in self.components: raise AssertionError(f"Component with name {component.name} already exists. Please rename the component.") self.components[component.name] = component def add_wiring(self, from_component: str, from_output: str, to_component: str, to_input: str): """ Creates a one-to-one connection between the input and output of two components. The method checks if the connection is allowed with the following requirements: - The components exist with the given inputs/outputs - The given inputs and outputs are not connected to any wire - The types of the connected input and output are compatible Args: from_component (str): Name of the component which provides data to the `to_component` from_output (str): Name of the output variable of the `from_component` to_component (str): Name of the component which consumes data from the `from_component` to_input (str): Name of the input variable of the `to_component` """ if from_component not in self.components: raise AssertionError(f"No component named '{from_component}'") if to_component not in self.components: raise AssertionError(f"No component named '{to_component}'") if from_output not in self.components[from_component].outputs: raise AssertionError(f"Component '{from_component}' has no output named '{from_output}'") if to_input not in self.components[to_component].inputs: raise AssertionError(f"Component '{to_component}' has no input named '{to_input}'") if self.get_wire_for_input(to_component, to_input) is not None: raise AssertionError(f"Input '{to_input}' of component '{to_component}' is already wired") _output_type = self.components[from_component].outputs[from_output]["type"] _input_type = self.components[to_component].inputs[to_input]["type"] if _output_type != _input_type: raise AssertionError("Output and input types do not match") wire_hash = self._wire_hash_string.format(from_component, from_output, to_component, to_input) self.wiring[wire_hash] = { "fromComponent": from_component, "fromOutput": from_output, "toComponent": to_component, "toInput": to_input, } def get_wire_for_output(self, component_name: str, output_name: str) -> Optional[dict]: """ Searches for the wire which connects a component with `component_name` as data provider through its output with name output_name. Args: component_name (str): Name of the data provider component. output_name (str): Name of the output variable of `component_name`. Returns: Optional[dict]: Wire which contains the data provider and receiver with their names and the names of their variables. """ wires = [x for x in self.wiring.values() if x["fromComponent"] == component_name and x["fromOutput"] == output_name] return wires[0] if wires else None def get_wire_for_input(self, component_name: str, input_name: str) -> Optional[dict]: """ Searches for the wire which connects a component with `component_name` as data consumer through its input with name `input_name`. Args: component_name (str): Name of the data consumer component. input_name (str): Name of the input variable of `component_name`. Returns: Optional[dict]: Wire which contains the data provider and receiver with their names and the names of their variables. """ wires = [x for x in self.wiring.values() if x["toComponent"] == component_name and x["toInput"] == input_name] return wires[0] if wires else None def delete_input_wire(self, component: str, variable: str, with_input: bool = True) -> None: """ Deletes an existing connection between two components. The connection must be given with the name of the consumer component and its input variable. If an inter signal alignment reference variable is affected it cannot be deleted. By default, the input variable will be also deleted. Args: component (str): Name of the component which has the input given the name variable variable (str): Name of the input variable on the component which connected by the wire with_input (bool, optional): If set, the input variable will be also deleted from the component. Defaults to True. Raises: AssertionError: When the variable acts as inter signal alignment reference, it cannot be deleted, and an `AssertionError` will be raised. """ wire = self.get_wire_for_input(component, variable) if wire is None: raise AssertionError(f"There is no wiring for input '{variable}' of component '{component}'") if variable in self.timeshift_reference: raise AssertionError("Inter signal alignment reference variables can not be deleted.") wire_hash = self._wire_hash_string.format(wire['fromComponent'], wire['fromOutput'], wire['toComponent'], wire['toInput']) self.wiring.pop(wire_hash) if with_input: self.components[component].delete_input(variable) def add_dependencies(self, packages: list) -> None: """ @Deprecated, reason: components can have different Python versions and/or platform, therefore it's better to specify dependencies on a case-by-case basis. Collects the given Python packages with their versions from the executing Python environment and add them to all components of type `PythonComponent`. This step is necessary in order to execute the pipeline configuration on the Edge side. The method can be called multiple times but each time the previously-collected dependencies are cleared. The reason for this is to ensure a consistent dependency list for the `requirements.txt` file when the package is saved. Args: packages (list): List of the necessary python packages to execute the script defined by self.entrypoint """ python_components = [self.components[name] for name in self.components if type(self.components[name]) is PythonComponent] for component in python_components: component.add_dependencies(packages) def set_timeshifting_periodicity(self, periodicity: int) -> None: """ Enables inter-signal alignment with the given sampling period. With inter-signal alignment enabled, the AI Inference Server collects data for different input variables before it triggers the model. By default, `startingPoint` property is set to `First timestamp`, which means that inter-signal alignment is started at the first incoming value for any input variable. This property can be changed to `Signal reference` by adding inter-signal alignment reference variables via the `add_timeshifting_reference(..)` method. In this case, inter-signal alignment is started when the first value arrives for the defined input variables. Args: periodicity (int): Periodicity time in milliseconds for the AI Inference Server to perform inter-signal alignment. Valid range is [10, 2^31). """ periodicity = int(periodicity) if periodicity not in range(10, int(math.pow(2, 31))): raise AssertionError("Inter signal alignment periodicity must be an integer and in range [10, 2^31)") self.periodicity = periodicity _logger.info(f"Inter signal alignment periodicity has been set to {self.periodicity}.") def add_timeshifting_reference(self, reference: str) -> None: """ Enables signal alignment mode `Signal reference` by declaring input variables as reference variables. Args: reference (str): Variable name to be added to `self.timeshift_reference` list. """ if reference not in [i.variableName for i in self.inputs]: raise AssertionError(f"There is no input variable defined with name '{reference}'") if reference in self.timeshift_reference: _logger.warning(f"Reference variable with name '{reference}' has been already added.") return self.timeshift_reference.append(reference) def remove_timeshifting_reference(self, reference: str) -> None: """ Removes previously-defined inter-signal alignment reference variables. If no reference variables remain, the `startingPoint` will be `First timestamp`. Args: reference (str): Variable name to be removed from `self.timeshift_reference` list. """ if reference not in self.timeshift_reference: raise AssertionError(f"Reference variable with name {'reference'} does not exist.") self.timeshift_reference.remove(reference) def set_cycle_time(self, cycle_time_usec: int | None): """ Sets the cycle time for the pipeline. Args: cycle_time_usec (int | None): Cycle time in microseconds. Must be a positive integer between 1_000 and 1_000_000. """ self.cycle_time = cycle_time_usec self._check_cycle_time() def get_pipeline_config(self) -> dict: """ Saves the information on the composed pipeline configuration package into a YAML file. This YAML file describes the components and the data flow between them for the AI Inference Server. The file is created in the `destination` folder with name `pipeline_config.yml` """ version = self.save_version if self.save_version is not None else self.init_version metric_fields = [(component_name, field) for component_name, component in self.components.items() if isinstance(component, PythonComponent) for field in component.metrics.keys()] pipeline_inputs = [ i.__dict__(self.components[i.componentName].inputs) for i in self.inputs ] pipeline_outputs = [ {**o.__dict__(self.components[o.componentName].outputs), 'metric': False} for o in self.outputs ] pipeline_outputs += [{ 'name': field, 'type': 'String', 'metric': True, 'topic': f"/siemens/edge/aiinference/{self.name}/{version}/metrics/{component_name}/{field}", } for component_name, field in metric_fields] pipeline_dag = [{ 'source': f'Databus.{i.variableName}', 'target': f'{i.componentName}.{i.variableName}', } for i in self.inputs] pipeline_dag += [{ 'source': f"{wire['fromComponent']}.{wire['fromOutput']}", 'target': f"{wire['toComponent']}.{wire['toInput']}", } for wire in self.wiring.values()] pipeline_dag += [{ 'source': f'{o.componentName}.{o.variableName}', 'target': f'Databus.{o.variableName}', } for o in self.outputs] pipeline_dag += [{ 'source': f'{component_name}.{field}', 'target': f'Databus.{field}', } for component_name, field in metric_fields] config_yml_content = { 'fileFormatVersion': '1.2.0', 'dataFlowPipelineInfo': { 'author': self.author, 'createdOn': datetime.now(), 'dataFlowPipelineVersion': version, 'description': self.desc if self.desc else 'Created by AI SDK', 'projectName': self.name, 'packageId': str(self.package_id) }, 'dataFlowPipeline': { 'components': [component._to_dict() for component in self.components.values()], 'pipelineDag': pipeline_dag, 'pipelineInputs': pipeline_inputs, 'pipelineOutputs': pipeline_outputs, }, 'packageType': 'full' } if len(self.parameters) > 0: config_yml_content["dataFlowPipeline"]["pipelineParameters"] = [ param.__param_dict__() for param in self.parameters ] if self.cycle_time is not None: config_yml_content["dataFlowPipeline"]["cycle_time"] = self.cycle_time return config_yml_content def save_pipeline_config(self, destination): """ Saves the information about the composed pipeline configuration package into a YAML file. This YAML file describes the components and the data flow between them for AI Inference Server. The file will be created in the `destination` folder with name `pipeline_config.yml` Args: destination (path-like): Path of the `destination` directory. """ with open(Path(destination) / PIPELINE_CONFIG, "w") as f: yaml.dump(self.get_pipeline_config(), f) def get_datalink_metadata(self) -> dict: """ The method generates metadata information based on available information. Returns: dict: Dictionary with the necessary information for the AI Inference Server. """ timeshifting = { "id": None, "enabled": False, "periodicity": self.periodicity, "startingPoint": None, } if self.periodicity is not None: timeshifting["enabled"] = True timeshifting["startingPoint"] = 'First timestamp' if len(self.timeshift_reference) > 0: timeshifting["startingPoint"] = 'Signal reference' exported_metadata = { "fileFormatVersion": "1.0.0", "id": None, "version": None, "createdOn": datetime.now(), "updatedOn": datetime.now(), "timeShifting": timeshifting, "inputs": [ { 'name': input.variableName, 'mapping': None, 'timeShiftingReference': input.variableName in self.timeshift_reference, 'type': self.components[input.componentName].inputs[input.variableName]['type'] } for input in self.inputs ] } return exported_metadata def save_datalink_metadata(self, destination): """ Saves metadata for pipeline input variables. This method saves metadata for the AI Inference Server into a YAML file. This metadata determines how the AI Inference Server feeds input to the pipeline, especially inter-signal alignment. The file is created in the `destination` folder with the name `datalink_metadata.yml` Args: destination (path-like): Path of the destination directory. """ with open(Path(destination) / DATALINK_METADATA, "w") as f: yaml.dump(self.get_datalink_metadata(), f) def save_telemetry_data_if_consented(self, destination: Path): try: Path(destination / TELEMETRY_YAML).unlink(missing_ok=True) # checking whether the user has made a statement about telemetry collection if is_telemetry_allowed(): self.save_telemetry_data(destination) except RuntimeError as consent_is_undecided: raise consent_is_undecided def save_telemetry_data(self, destination: Path): """ Save telemetry data to a specified destination. Args: destination (Path): The path where the telemetry data should be saved. """ telemetry_path = destination / TELEMETRY_YAML telemetry_data = {} try: consent_allowed = is_telemetry_allowed() except RuntimeError: consent_allowed = "undecided" try: consent_data, _ = read_telemetry_consent_file() except BaseException: consent_data = {} telemetry_data["telemetry_consent"] = {} telemetry_data["telemetry_consent"][CONSENT_KEY] = consent_allowed telemetry_data["telemetry_consent"][CONSENT_KEY_SDK] = consent_data.get(CONSENT_KEY_SDK, 'unknown') telemetry_data["telemetry_consent"][CONSENT_KEY_TS] = consent_data.get(CONSENT_KEY_TS, 'unknown') telemetry_data["telemetry_consent"][CONSENT_KEY_METHOD] = consent_data.get(CONSENT_KEY_METHOD, get_consent_source()) telemetry_data["platform"] = {} telemetry_data["platform"]["os"] = platform.system() telemetry_data["platform"]["release"] = platform.release() telemetry_data["platform"]["python_version"] = platform.python_version() _logger.info(f"locals: {locals()}") telemetry_data["environment"] = {} env_info = "Jupyter" if any(k for k in locals() if k in ["__IPYTHON__", "get_ipython"]) else MSG_NOT_FOUND env_info = "GitLab CI/CD" if "GITLAB_CI" in os.environ else env_info env_info = "Azure DevOps Pipelines" if "TF_BUILD" in os.environ else env_info env_info = "GitHub Actions" if "GITHUB_ACTIONS" in os.environ else env_info telemetry_data["environment"]["info"] = env_info telemetry_data["industrial_ai"] = {} try: telemetry_data["industrial_ai"]["simaticai"] = importlib_metadata.version("simaticai") except importlib_metadata.PackageNotFoundError: telemetry_data["industrial_ai"]["simaticai"] = MSG_NOT_FOUND _logger.debug("simaticai package not found") telemetry_data["pipeline"] = {} telemetry_data["pipeline"]["python_versions"] = list(set(self.components[component].python_version for component in self.components if isinstance(self.components[component], PythonComponent))) telemetry_data["pipeline"]["file_extensions"] = list(set(f.suffix for f in Path(destination).rglob("*") if f.suffix not in ["", ".zip", ".yml", ".yaml", ".html"])) with open(telemetry_path, 'w') as telemetry_file: yaml.dump(telemetry_data, telemetry_file) def validate(self, destination: str | Path = "."): """ Validates whether the package configuration is compatible with the expected runtime environment. The method verifies: - If the package has at least one component - If all wires create connections between existing components and their variables - If metadata is defined and valid. - If a package with the same name already exists in the `destination` folder. In this case a warning message appears and the `save(..)` method overwrites the existing package. - If the package has multiple components and if they are using the same Python version Args: destination (str, optional): Path of the expected destination folder. Defaults to ".". """ version = self.save_version if self.save_version is not None else self.init_version if len(self.components) < 1: raise AssertionError("The package must have at least one component.") for p_output in self.outputs: if self.components[p_output.componentName].batch.outputBatch: raise AssertionError(f"The component '{p_output.componentName}' has pipeline output defined with variable name '{p_output.variableName}'. \ None of component with pipeline output is allowed to provide batch output.") for wire_hash in self.wiring.copy(): wire = self.wiring[wire_hash] self._check_wiring(wire, wire_hash) pipeline_inputs = [p_input.variableName for p_input in self.inputs] pipeline_outputs = [p_output.variableName for p_output in self.outputs] if any(variable in pipeline_outputs for variable in pipeline_inputs): conflicts = set(pipeline_inputs).intersection(set(pipeline_outputs)) raise AssertionError(f"Pipeline input and output variables must be unique. Conflicting variables: {conflicts}") self._check_timeshifting() self._check_cycle_time() self._check_supported_types_for_connections() package_path = Path(destination) / f"{self.name}_{version}".replace(" ", "-") if package_path.is_dir(): _logger.warning(f"Target folder ({package_path}) already exists! Unless changing the package name the package could be invalid and your files will be overwritten!") python_versions = set() for component in self.components: self.components[component].validate() if isinstance(self.components[component], PythonComponent): python_versions.add(self.components[component].python_version) if (1 < len(python_versions)): _logger.warning("The use of multiple python version in a single pipeline is not recommended. We recommend using only one of the supported versions, which are Python 3.11 or 3.12.") _logger.info(f"Package '{self.name}' is valid and ready to save.") def _check_timeshifting(self): if len(self.timeshift_reference) > 0 and self.periodicity is None: raise AssertionError("When using inter signal alignment reference variables, the periodicity must be set.") def _check_supported_types_for_connections(self): for p_input in self.inputs: variable_type = self.components[p_input.componentName].inputs[p_input.variableName]['type'] warnings, errors = p_input.check_connection_type_compatibility(variable_type) warnings_str = ", ".join(warnings) errors_str = ", ".join(errors) if warnings: _logger.warning(f"Input variable `{p_input.variableName}` of component `{p_input.componentName}` is of type " f"`{variable_type}` which is not supported for connection type `{p_input.connection.cptype.name}`. " f"Issues: {warnings_str}.") if errors: _logger.error(f"Input variable `{p_input.variableName}` of component `{p_input.componentName}` is of type " f"`{variable_type}` which is not supported for connection type `{p_input.connection.cptype.name}`. " f"Issues: {errors_str}.") for p_output in self.outputs: variable_type = self.components[p_output.componentName].outputs[p_output.variableName]['type'] warnings, errors = p_output.check_connection_type_compatibility(variable_type) warnings_str = ", ".join(warnings) errors_str = ", ".join(errors) if warnings: _logger.warning(f"Output variable `{p_output.variableName}` of component `{p_output.componentName}` is of type " f"`{variable_type}` which is not supported for connection type `{p_output.connection.cptype.name}`. " f"Issues: {warnings_str}.") if errors: _logger.error(f"Output variable `{p_output.variableName}` of component `{p_output.componentName}` is of type " f"`{variable_type}` which is not supported for connection type `{p_output.connection.cptype.name}`. " f"Issues: {errors_str}.") for p_param in self.parameters: if isinstance(p_param, SecretPipelineParameter): continue if not p_param.topicBased: continue if not p_param.has_valid_connection(): _logger.error(f"Pipeline parameter `{p_param.name}` is of type `{p_param.dtype}` which is not supported for connection type `{p_param.connection.cptype.name}`.") def _check_wiring(self, wire, wire_hash): error_messages = [] if wire['fromComponent'] not in self.components: error_messages.append(f"From component {wire['fromComponent']} does not exist") if wire['toComponent'] not in self.components: error_messages.append(f"To component {wire['toComponent']} does not exist") if wire['fromOutput'] not in self.components[wire['fromComponent']].outputs: error_messages.append(f"Output variable {wire['fromOutput']} does not exist on component {wire['fromComponent']}") if wire['toInput'] not in self.components[wire['toComponent']].inputs: error_messages.append(f"Input variable {wire['toInput']} does not exist on component {wire['toComponent']}") if len(error_messages) == 0: from_type_ = self.components[wire['fromComponent']].outputs[wire['fromOutput']]['type'] to_type_ = self.components[wire['toComponent']].inputs[wire['toInput']]['type'] if from_type_ != to_type_: error_messages.append(f"The types of input and output variables does not match for wiring {wire_hash}.") if len(error_messages) > 0: self.wiring.pop(wire_hash) error_messages.append("The wire has been deleted, please check the variables and re-create the connection.") raise AssertionError(error_messages.__str__()) def _check_cycle_time(self) -> bool: if self.cycle_time is None: return True if not isinstance(self.cycle_time, int) or self.cycle_time < 1: _logger.error(cycle_time_error_message) return False if self.cycle_time < 1_000 or 1_000_000 < self.cycle_time: _logger.warning(cycle_time_error_message) return False return True def save(self, destination: str | Path = ".", package_id: Optional[uuid.UUID] = None, version: Optional[str] = None) -> Path: """ @Deprecated, reason: only edge package generation will be supported in the future. Use export instead. Saves the assembled package in a zip format. The name of the file is defined as `{package_name}_{package_version}.zip`. If a file with such a name already exists in the `destination` folder, it gets overwritten and a warning message appears. The package is also available as a subfolder on the destination path with the name `{package_name}_{package_version}`. If the assembled content does not meet the expected one, this content can be changed and simply packed into a zip file. The package contains files and folders in the following structure: - Package folder with name `{package_name}_{package_version}` - `datalink-metadata.yml` - `pipeline-config.yml` - Component folder with name `{component_name}` When the component is a `PythonComponent`, this folder contains: - `requirements.txt` - Entrypoint script defined by the entrypoint of the component - Extra files as added to the specified folders - Source folder with name `src` with necessary python scripts If a package ID is specified, and a package with the same ID and version is already present in the `destination` folder, an error is raised. Args: destination (str, optional): Target directory for saving the package. Defaults to ".". package_id (UUID): The optional package ID. If None, a new UUID is generated. """ try: # checking whether the user has made a statement about telemetry collection is_telemetry_allowed() except RuntimeError as consent_is_undecided: raise consent_is_undecided self._set_save_version_and_package_id(Path(destination), package_id, version) destination = Path(destination) self.validate(destination) name = self.name.replace(" ", "-") package_name = f"{name}_{self.save_version}" destination = destination / package_name destination.mkdir(parents=True, exist_ok=True) for component in self.components: self.components[component].save(destination, False) if isinstance(self.components[component], PythonComponent): self.report_writer.add_direct_dependencies(self.components[component].name, self.components[component].python_dependencies.dependencies) self.save_datalink_metadata(destination) self.save_pipeline_config(destination) p_page.save_readme_html(self, destination) self.save_telemetry_data_if_consented(destination) zip_destination = shutil.make_archive( base_name=str(destination.parent / package_name), format='zip', root_dir=destination.parent, base_dir=package_name, verbose=True, logger=_logger) return Path(zip_destination) def _set_save_version_and_package_id(self, destination: Path, package_id: Optional[uuid.UUID], version: Optional[str]): previous_versions_and_ids = self._get_versions_and_package_ids_of_existing_packages(destination) previous_versions, previous_package_ids = zip(*previous_versions_and_ids) if previous_versions_and_ids else ([], []) # if package id is provided, we use that if package_id is not None: self.package_id = package_id # if not, we auto-generate a package id else: previous_package_ids_set = {pkg_id for pkg_id in previous_package_ids if pkg_id is not None} if self.package_id is not None: previous_package_ids_set.add(self.package_id) if len(previous_package_ids_set) == 0: self.package_id = uuid.uuid4() elif len(previous_package_ids_set) == 1: self.package_id = previous_package_ids_set.pop() else: _logger.error(f"Multiple package IDs found in the destination folder: {previous_package_ids_set}. Set a package ID.") raise RuntimeError(f"Multiple package IDs found in the destination folder: {previous_package_ids_set}. Set a package ID.") # Preference #1: use the provided version if version is not None: self.save_version = version # Preference #2: use the version set at init time elif self.init_version is not None: self.save_version = self.init_version # Preference #3: auto-generate version else: previous_decimal_versions_set = {int(v) for v in previous_versions if v is not None and v.isdecimal()} if len(previous_decimal_versions_set) == 0: self.save_version = "1" else: self.save_version = str(max(previous_decimal_versions_set) + 1) # checking if the package zip already exists name = self.name.replace(" ", "-") package_name = f"{name}_{self.save_version}" package_file = destination / f"{package_name}.zip" if package_file.exists(): _logger.warning(f"Target package with version '{self.save_version}' already exists: '{package_file}. The package will be overwritten.") edge_package_file = destination / f"{name}-edge_{self.save_version}.zip" if edge_package_file.exists(): _logger.warning(f"Target package with version '{self.save_version}' already exists: '{edge_package_file}. The package will be overwritten.") self.version = self.save_version def _get_versions_and_package_ids_of_existing_packages(self, destination: Path) -> List[Tuple[str, Optional[uuid.UUID]]]: package_versions_and_ids = [] for file in destination.glob(f"{self.name.replace(' ', '-')}*.zip"): with zipfile.ZipFile(file) as zip_file: config_path = next(f for f in zip_file.namelist() if f.endswith("pipeline_config.yml")) with zip_file.open(config_path) as config_file: config = yaml.load(config_file, Loader=yaml.SafeLoader) pipeline_info = config.get("dataFlowPipelineInfo", {}) name = pipeline_info.get("projectName", None) if name is None or name != self.name: continue version = pipeline_info.get("dataFlowPipelineVersion", None) package_id = pipeline_info.get("packageId", None) package_id = uuid.UUID(package_id) if package_id is not None else None package_versions_and_ids.append((version, package_id)) return package_versions_and_ids def export(self, destination: str | Path = ".", package_id: Optional[uuid.UUID] = None, version: Optional[str] = None) -> Path: """ Export a runnable pipeline package. Args: destination (str): optional target directory for saving the package. Defaults to ".". package_id (UUID): optional package ID. If None, a new UUID is generated. version (str): optional version. If None, an automatic version number is generated. """ try: # checking whether the user has made a statement about telemetry collection is_telemetry_allowed() except RuntimeError as consent_is_undecided: raise consent_is_undecided destination = Path(destination) config_package = None try: config_package = self.save(destination, package_id, version) runtime_package = convert_package(config_package, self.report_writer) return runtime_package finally: if config_package is not None: Path(config_package).unlink(missing_ok=True) def _add_parameter(self, parameter: PipelineParameter | SecretPipelineParameter): if any(param.name == parameter.name for param in self.parameters): raise AssertionError(f"Parameter with name '{parameter.name}' already exists in the pipeline.") if parameter is None: raise AssertionError("Parameter cannot be None.") self.parameters.append(parameter) def add_secret_parameter(self, name: str, desc: str | None = None): """ Adds a secret parameter to the pipeline configuration. Args: name (str): Name of the secret parameter. Must contain only letters, digits, underscores, and hyphens. Cannot start with `__AI_IS_` prefix. desc (str, optional): Description of the secret parameter. Must be a string between 3 and 60 characters if provided. Raises ValueError if the conditions for name or description are not met. """ self._add_parameter(SecretPipelineParameter(name=name, description=desc)) def add_parameter(self, name: str, default_value, type_name: str = "String", topic_based: bool = False, desc: str = None): """ Adds a parameter to the pipeline configuration, which alters the behavior of the pipeline. The parameter's default value and its properties are saved in the pipeline configuration and the value of the parameter can later be changed on AI Inference Server. Args: name (str): Name of the parameter desc (str): Description of the parameter (optional) type_name (str, optional): Data type of the parameter. Defaults to "String". default_value (str): Default value of the parameter topic_based (bool, optional): If true, the parameter can be updated from a message queue. Raises: ValueError: When: - the default value of the parameter is not of the specified data type (`type_name`) or - the specified data type itself is not an allowed data type (not a part of `parameter_types` dict) or - the specified data type is not given in the right format or - the type of the given `topic_based` parameter is not `bool`. - the name of the parameter starts with `__AI_IS_` prefix. These are reserved parameters by AI Inference Server """ self._add_parameter(PipelineParameter( name=name, defaultValue=default_value, dtype=type_name, topicBased=topic_based, description=desc)) ``` ### `__init__(name, version=None, desc='')` A newly initialized `Pipeline` will contain no `Component` or wire, just its name and version will be set. The name and version will define together the name of the zip file when the package is saved. Parameters: | Name | Type | Description | Default | | --------- | ----- | ------------------------------ | ---------- | | `name` | `str` | Name of the package | *required* | | `desc` | `str` | Package description (optional) | `''` | | `version` | `str` | Version of the package | `None` | Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/pipeline.py` ```python def __init__(self, name: str, version: Optional[str] = None, desc: str = ""): """ A newly initialized `Pipeline` will contain no `Component` or wire, just its name and version will be set. The name and version will define together the name of the zip file when the package is saved. Args: name (str): Name of the package desc (str): Package description (optional) version (str): Version of the package """ super().__init__(name, version, desc) self.report_writer = PipelineReportWriter() report_writer_handler = ReportWriterHandler(self.report_writer) _logger.addHandler(report_writer_handler) _python_dependencies_logger.addHandler(report_writer_handler) _wheelhouse_logger.addHandler(report_writer_handler) ``` ### `from_components(components, name, version=None, desc='')` Creates a pipeline configuration from the given components. The components are linked in a linear sequence with inputs and outputs auto-wired based on the name of the inputs and outputs of the components. The inputs of the first component will be wired as the pipeline inputs and the outputs of the last component will be wired as the pipeline outputs. The components must have unique names. Two or more versions of the same component can not be packaged simultaneously without renaming them. Parameters: | Name | Type | Description | Default | | ------------ | ------ | ----------------------------------------------- | ---------- | | `components` | `list` | List of PythonComponents | *required* | | `name` | `str` | Name of the pipeline | *required* | | `version` | `str` | Version information of the pipeline. (Optional) | `None` | Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/pipeline.py` ```python @staticmethod def from_components(components: list, name: str, version: Optional[str] = None, desc: str = "") -> "Pipeline": """ Creates a pipeline configuration from the given components. The components are linked in a linear sequence with inputs and outputs auto-wired based on the name of the inputs and outputs of the components. The inputs of the first component will be wired as the pipeline inputs and the outputs of the last component will be wired as the pipeline outputs. The components must have unique names. Two or more versions of the same component can not be packaged simultaneously without renaming them. Args: components (list): List of PythonComponents name (str): Name of the pipeline version (str): Version information of the pipeline. (Optional) Returns: Pipeline: Pipeline object with the auto-wired components """ pipeline = Pipeline(name, version, desc=desc) first_component = components[0] pipeline.add_component(first_component) pipeline.inputs = [PipelineVariable(first_component.name, component_input) for component_input in first_component.inputs] pipeline.outputs = [PipelineVariable(first_component.name, output) for output in first_component.outputs] for component in components[1:]: pipeline.add_component(component) for t_output in pipeline.outputs: try: pipeline.add_wiring(t_output.componentName, t_output.variableName, component.name, t_output.variableName) except Exception as e: _logger.warning(f"Output variable {t_output.componentName}.{t_output.variableName} couldn't be auto-wired.\nCause: {e}") unwired_variables = [f'{component.name}.{x}' for x in component.inputs if not any(s.endswith(f'{component.name}.{x}') for s in pipeline.wiring)] if len(unwired_variables) > 0: for variable in unwired_variables: _logger.warning(f"Input variable {variable} couldn't be auto-wired.\n") pipeline.outputs = [PipelineVariable(component.name, output) for output in component.outputs] # ImageSet is not allowed in pipeline outputs for i, o in enumerate(pipeline.outputs): if pipeline.components[o.componentName].outputs[o.variableName]["type"].lower() == 'imageset': _logger.warning(f"Output variable {o.componentName}.{o.variableName} of type 'ImageSet' can not be added as pipeline output.") pipeline.outputs.pop(i) return pipeline ``` ### `__repr__()` Textual representation of the configured package. The method shows the `Components` with their inputs, outputs and parameters as well as the wiring between these `Components`. Returns: | Type | Description | | ----- | ---------------------------------------------- | | `str` | \[str\]: Textual representation of the package | Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/pipeline.py` ```python def __repr__(self) -> str: """ Textual representation of the configured package. The method shows the `Components` with their inputs, outputs and parameters as well as the wiring between these `Components`. Returns: [str]: Textual representation of the package """ version = self.save_version if self.save_version is not None else self.init_version text = f"[{self.__class__.__name__}] {self.name} ({version})\n" if self.desc != "": text += f"{self.desc}\n" if len(self.parameters) > 0: text += "\nPipeline Parameters:\n" for parameter in self.parameters: text += f"- {parameter.name} ({parameter.dtype}, default: '{parameter.defaultValue}'){(': ' + parameter.description) if parameter.description is not None else ''}\n" if len(self.inputs) > 0: text += "\nPipeline Inputs:\n" for p_input in self.inputs: c_input = self.components[p_input.componentName].inputs[p_input.variableName] text += f"> {p_input.variableName} ({c_input['type']}){': ' + c_input['desc'] if c_input.get('desc') is not None else ''}\n" if len(self.outputs) > 0: text += "\nPipeline Outputs:\n" for p_output in self.outputs: c_output = self.components[p_output.componentName].outputs[p_output.variableName] text += f"< {p_output.variableName} ({c_output['type']}){': ' + c_output['desc'] if c_output.get('desc') is not None else ''}\n" metrics = [(name, metric, component_name) for component_name, component in self.components.items() if isinstance(component, PythonComponent) for name, metric in component.metrics.items()] if len(metrics) > 0: text += "\nMetrics:\n" for name, metric, _ in metrics: text += f"< {name}{': ' + metric['desc'] if metric.get('desc') is not None else ''}\n" if len(self.wiring) > 0: text += "\nI/O Wiring:\n" for p_input in self.inputs: text += f" {p_input.variableName} -> {p_input.componentName}.{p_input.variableName}\n" for wire_hash in self.wiring: text += f" {wire_hash}\n" for p_output in self.outputs: text += f" {p_output.componentName}.{p_output.variableName} -> {p_output.variableName}\n" for name, metric, component_name in metrics: text += f" {component_name}.{name} -> {name}\n" if self.periodicity is not None: text += "\nTimeshifting:\n" text += f" Periodicity: {self.periodicity} ms\n" if len(self.timeshift_reference) > 0: text += " References:\n" for ref in self.timeshift_reference: text += f" - {ref}\n" if self.cycle_time is not None: text += f"\nCycle Time: {self.cycle_time} ms\n" for component in self.components.values(): text += "\n" + component.__repr__() return text ``` ### `add_input(component, variable)` Defines an input variable on the given component as a pipeline input. Parameters: | Name | Type | Description | Default | | ----------- | ----- | -------------------------- | ---------- | | `component` | `str` | Name of the component | *required* | | `variable` | `str` | Name of the input variable | *required* | Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/pipeline.py` ```python def add_input(self, component, variable): """ Defines an input variable on the given component as a pipeline input. Args: component (str): Name of the component variable (str): Name of the input variable """ try: _ = self.components[component].inputs[variable] except KeyError: raise AssertionError("The component with input variable must exist in the pipeline.") if self.inputs is None: self.inputs = [] if self._get_io_index(self.inputs, variable) is not None: raise AssertionError("The pipeline input already exists.") self.inputs.append(PipelineVariable(componentName=component, variableName=variable)) ``` ### `delete_input(component, variable)` Deletes a pipeline input. Parameters: | Name | Type | Description | Default | | ----------- | ----- | -------------------------- | ---------- | | `component` | `str` | Name of the component | *required* | | `variable` | `str` | Name of the input variable | *required* | Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/pipeline.py` ```python def delete_input(self, component: str, variable: str): """ Deletes a pipeline input. Args: component (str): Name of the component variable (str): Name of the input variable """ if (idx := self._get_io_index(self.inputs, variable)) is None: raise AssertionError("The pipeline input does not exist.") self.inputs.pop(idx) ``` ### `add_output(component, variable)` Defines an output variable on the given component as a pipeline output. Parameters: | Name | Type | Description | Default | | ----------- | ----- | --------------------------- | ---------- | | `component` | `str` | Name of the component | *required* | | `variable` | `str` | Name of the output variable | *required* | Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/pipeline.py` ```python def add_output(self, component, variable): """ Defines an output variable on the given component as a pipeline output. Args: component (str): Name of the component variable (str): Name of the output variable """ try: output = self.components[component].outputs[variable] except KeyError: raise AssertionError("The component with output variable must exist in the pipeline.") if self.outputs is None: self.outputs = [] if self._get_io_index(self.outputs, variable) is not None: raise AssertionError("The pipeline output already exists.") if output["type"].lower() == 'imageset': _logger.warning("Output variables of type 'ImageSet' can not be added as pipeline outputs.") self.outputs.append(PipelineVariable(componentName=component, variableName=variable)) ``` ### `delete_output(component, variable)` Deletes a pipeline output. Parameters: | Name | Type | Description | Default | | ----------- | ----- | --------------------------- | ---------- | | `component` | `str` | Name of the component | *required* | | `variable` | `str` | Name of the output variable | *required* | Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/pipeline.py` ```python def delete_output(self, component: str, variable: str): """ Deletes a pipeline output. Args: component (str): Name of the component variable (str): Name of the output variable """ if (idx := self._get_io_index(self.outputs, variable)) is None: raise AssertionError("The pipeline output does not exist.") self.outputs.pop(idx) ``` ### `add_component(component)` Adds a `Component` to the pipeline configuration without any connection. The component must have a unique name. Two or more versions of the same component can not be added to the same pipeline with the same component name. Parameters: | Name | Type | Description | Default | | ----------- | ----------- | --------------------- | ---------- | | `component` | `Component` | Component to be added | *required* | Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/pipeline.py` ```python def add_component(self, component: Component): """ Adds a `Component` to the pipeline configuration without any connection. The component must have a unique name. Two or more versions of the same component can not be added to the same pipeline with the same component name. Args: component (Component): `Component` to be added """ if component.name in self.components: raise AssertionError(f"Component with name {component.name} already exists. Please rename the component.") self.components[component.name] = component ``` ### `add_wiring(from_component, from_output, to_component, to_input)` Creates a one-to-one connection between the input and output of two components. The method checks if the connection is allowed with the following requirements: - The components exist with the given inputs/outputs - The given inputs and outputs are not connected to any wire - The types of the connected input and output are compatible Parameters: | Name | Type | Description | Default | | ---------------- | ----- | ----------------------------------------------------------------- | ---------- | | `from_component` | `str` | Name of the component which provides data to the to_component | *required* | | `from_output` | `str` | Name of the output variable of the from_component | *required* | | `to_component` | `str` | Name of the component which consumes data from the from_component | *required* | | `to_input` | `str` | Name of the input variable of the to_component | *required* | Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/pipeline.py` ```python def add_wiring(self, from_component: str, from_output: str, to_component: str, to_input: str): """ Creates a one-to-one connection between the input and output of two components. The method checks if the connection is allowed with the following requirements: - The components exist with the given inputs/outputs - The given inputs and outputs are not connected to any wire - The types of the connected input and output are compatible Args: from_component (str): Name of the component which provides data to the `to_component` from_output (str): Name of the output variable of the `from_component` to_component (str): Name of the component which consumes data from the `from_component` to_input (str): Name of the input variable of the `to_component` """ if from_component not in self.components: raise AssertionError(f"No component named '{from_component}'") if to_component not in self.components: raise AssertionError(f"No component named '{to_component}'") if from_output not in self.components[from_component].outputs: raise AssertionError(f"Component '{from_component}' has no output named '{from_output}'") if to_input not in self.components[to_component].inputs: raise AssertionError(f"Component '{to_component}' has no input named '{to_input}'") if self.get_wire_for_input(to_component, to_input) is not None: raise AssertionError(f"Input '{to_input}' of component '{to_component}' is already wired") _output_type = self.components[from_component].outputs[from_output]["type"] _input_type = self.components[to_component].inputs[to_input]["type"] if _output_type != _input_type: raise AssertionError("Output and input types do not match") wire_hash = self._wire_hash_string.format(from_component, from_output, to_component, to_input) self.wiring[wire_hash] = { "fromComponent": from_component, "fromOutput": from_output, "toComponent": to_component, "toInput": to_input, } ``` ### `get_wire_for_output(component_name, output_name)` Searches for the wire which connects a component with `component_name` as data provider through its output with name output_name. Parameters: | Name | Type | Description | Default | | ---------------- | ----- | ---------------------------------------------- | ---------- | | `component_name` | `str` | Name of the data provider component. | *required* | | `output_name` | `str` | Name of the output variable of component_name. | *required* | Returns: | Type | Description | | ---------------- | ----------------------------------------------------------------------------------------------------------------------- | | `Optional[dict]` | Optional\[dict\]: Wire which contains the data provider and receiver with their names and the names of their variables. | Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/pipeline.py` ```python def get_wire_for_output(self, component_name: str, output_name: str) -> Optional[dict]: """ Searches for the wire which connects a component with `component_name` as data provider through its output with name output_name. Args: component_name (str): Name of the data provider component. output_name (str): Name of the output variable of `component_name`. Returns: Optional[dict]: Wire which contains the data provider and receiver with their names and the names of their variables. """ wires = [x for x in self.wiring.values() if x["fromComponent"] == component_name and x["fromOutput"] == output_name] return wires[0] if wires else None ``` ### `get_wire_for_input(component_name, input_name)` Searches for the wire which connects a component with `component_name` as data consumer through its input with name `input_name`. Parameters: | Name | Type | Description | Default | | ---------------- | ----- | --------------------------------------------- | ---------- | | `component_name` | `str` | Name of the data consumer component. | *required* | | `input_name` | `str` | Name of the input variable of component_name. | *required* | Returns: | Type | Description | | ---------------- | ----------------------------------------------------------------------------------------------------------------------- | | `Optional[dict]` | Optional\[dict\]: Wire which contains the data provider and receiver with their names and the names of their variables. | Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/pipeline.py` ```python def get_wire_for_input(self, component_name: str, input_name: str) -> Optional[dict]: """ Searches for the wire which connects a component with `component_name` as data consumer through its input with name `input_name`. Args: component_name (str): Name of the data consumer component. input_name (str): Name of the input variable of `component_name`. Returns: Optional[dict]: Wire which contains the data provider and receiver with their names and the names of their variables. """ wires = [x for x in self.wiring.values() if x["toComponent"] == component_name and x["toInput"] == input_name] return wires[0] if wires else None ``` ### `delete_input_wire(component, variable, with_input=True)` Deletes an existing connection between two components. The connection must be given with the name of the consumer component and its input variable. If an inter signal alignment reference variable is affected it cannot be deleted. By default, the input variable will be also deleted. Parameters: | Name | Type | Description | Default | | ------------ | ------ | ------------------------------------------------------------------------------------- | ---------- | | `component` | `str` | Name of the component which has the input given the name variable | *required* | | `variable` | `str` | Name of the input variable on the component which connected by the wire | *required* | | `with_input` | `bool` | If set, the input variable will be also deleted from the component. Defaults to True. | `True` | Raises: | Type | Description | | ---------------- | ----------------------------------------------------------------------------------------------------------------------- | | `AssertionError` | When the variable acts as inter signal alignment reference, it cannot be deleted, and an AssertionError will be raised. | Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/pipeline.py` ```python def delete_input_wire(self, component: str, variable: str, with_input: bool = True) -> None: """ Deletes an existing connection between two components. The connection must be given with the name of the consumer component and its input variable. If an inter signal alignment reference variable is affected it cannot be deleted. By default, the input variable will be also deleted. Args: component (str): Name of the component which has the input given the name variable variable (str): Name of the input variable on the component which connected by the wire with_input (bool, optional): If set, the input variable will be also deleted from the component. Defaults to True. Raises: AssertionError: When the variable acts as inter signal alignment reference, it cannot be deleted, and an `AssertionError` will be raised. """ wire = self.get_wire_for_input(component, variable) if wire is None: raise AssertionError(f"There is no wiring for input '{variable}' of component '{component}'") if variable in self.timeshift_reference: raise AssertionError("Inter signal alignment reference variables can not be deleted.") wire_hash = self._wire_hash_string.format(wire['fromComponent'], wire['fromOutput'], wire['toComponent'], wire['toInput']) self.wiring.pop(wire_hash) if with_input: self.components[component].delete_input(variable) ``` ### `add_dependencies(packages)` @Deprecated, reason: components can have different Python versions and/or platform, therefore it's better to specify dependencies on a case-by-case basis. Collects the given Python packages with their versions from the executing Python environment and add them to all components of type `PythonComponent`. This step is necessary in order to execute the pipeline configuration on the Edge side. The method can be called multiple times but each time the previously-collected dependencies are cleared. The reason for this is to ensure a consistent dependency list for the `requirements.txt` file when the package is saved. Parameters: | Name | Type | Description | Default | | ---------- | ------ | -------------------------------------------------------------------------------------- | ---------- | | `packages` | `list` | List of the necessary python packages to execute the script defined by self.entrypoint | *required* | Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/pipeline.py` ```python def add_dependencies(self, packages: list) -> None: """ @Deprecated, reason: components can have different Python versions and/or platform, therefore it's better to specify dependencies on a case-by-case basis. Collects the given Python packages with their versions from the executing Python environment and add them to all components of type `PythonComponent`. This step is necessary in order to execute the pipeline configuration on the Edge side. The method can be called multiple times but each time the previously-collected dependencies are cleared. The reason for this is to ensure a consistent dependency list for the `requirements.txt` file when the package is saved. Args: packages (list): List of the necessary python packages to execute the script defined by self.entrypoint """ python_components = [self.components[name] for name in self.components if type(self.components[name]) is PythonComponent] for component in python_components: component.add_dependencies(packages) ``` ### `set_timeshifting_periodicity(periodicity)` Enables inter-signal alignment with the given sampling period. With inter-signal alignment enabled, the AI Inference Server collects data for different input variables before it triggers the model. By default, `startingPoint` property is set to `First timestamp`, which means that inter-signal alignment is started at the first incoming value for any input variable. This property can be changed to `Signal reference` by adding inter-signal alignment reference variables via the `add_timeshifting_reference(..)` method. In this case, inter-signal alignment is started when the first value arrives for the defined input variables. Parameters: | Name | Type | Description | Default | | ------------- | ----- | --------------------------------------------------------------------------------------------------------------------------- | ---------- | | `periodicity` | `int` | Periodicity time in milliseconds for the AI Inference Server to perform inter-signal alignment. Valid range is \[10, 2^31). | *required* | Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/pipeline.py` ```python def set_timeshifting_periodicity(self, periodicity: int) -> None: """ Enables inter-signal alignment with the given sampling period. With inter-signal alignment enabled, the AI Inference Server collects data for different input variables before it triggers the model. By default, `startingPoint` property is set to `First timestamp`, which means that inter-signal alignment is started at the first incoming value for any input variable. This property can be changed to `Signal reference` by adding inter-signal alignment reference variables via the `add_timeshifting_reference(..)` method. In this case, inter-signal alignment is started when the first value arrives for the defined input variables. Args: periodicity (int): Periodicity time in milliseconds for the AI Inference Server to perform inter-signal alignment. Valid range is [10, 2^31). """ periodicity = int(periodicity) if periodicity not in range(10, int(math.pow(2, 31))): raise AssertionError("Inter signal alignment periodicity must be an integer and in range [10, 2^31)") self.periodicity = periodicity _logger.info(f"Inter signal alignment periodicity has been set to {self.periodicity}.") ``` ### `add_timeshifting_reference(reference)` Enables signal alignment mode `Signal reference` by declaring input variables as reference variables. Parameters: | Name | Type | Description | Default | | ----------- | ----- | ----------------------------------------------------------- | ---------- | | `reference` | `str` | Variable name to be added to self.timeshift_reference list. | *required* | Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/pipeline.py` ```python def add_timeshifting_reference(self, reference: str) -> None: """ Enables signal alignment mode `Signal reference` by declaring input variables as reference variables. Args: reference (str): Variable name to be added to `self.timeshift_reference` list. """ if reference not in [i.variableName for i in self.inputs]: raise AssertionError(f"There is no input variable defined with name '{reference}'") if reference in self.timeshift_reference: _logger.warning(f"Reference variable with name '{reference}' has been already added.") return self.timeshift_reference.append(reference) ``` ### `remove_timeshifting_reference(reference)` Removes previously-defined inter-signal alignment reference variables. If no reference variables remain, the `startingPoint` will be `First timestamp`. Parameters: | Name | Type | Description | Default | | ----------- | ----- | --------------------------------------------------------------- | ---------- | | `reference` | `str` | Variable name to be removed from self.timeshift_reference list. | *required* | Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/pipeline.py` ```python def remove_timeshifting_reference(self, reference: str) -> None: """ Removes previously-defined inter-signal alignment reference variables. If no reference variables remain, the `startingPoint` will be `First timestamp`. Args: reference (str): Variable name to be removed from `self.timeshift_reference` list. """ if reference not in self.timeshift_reference: raise AssertionError(f"Reference variable with name {'reference'} does not exist.") self.timeshift_reference.remove(reference) ``` ### `set_cycle_time(cycle_time_usec)` Sets the cycle time for the pipeline. Parameters: | Name | Type | Description | Default | | ----------------- | ----- | ----------- | ----------------------------------------------------------------------------------- | | `cycle_time_usec` | \`int | None\` | Cycle time in microseconds. Must be a positive integer between 1_000 and 1_000_000. | Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/pipeline.py` ```python def set_cycle_time(self, cycle_time_usec: int | None): """ Sets the cycle time for the pipeline. Args: cycle_time_usec (int | None): Cycle time in microseconds. Must be a positive integer between 1_000 and 1_000_000. """ self.cycle_time = cycle_time_usec self._check_cycle_time() ``` ### `get_pipeline_config()` Saves the information on the composed pipeline configuration package into a YAML file. This YAML file describes the components and the data flow between them for the AI Inference Server. The file is created in the `destination` folder with name `pipeline_config.yml` Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/pipeline.py` ```python def get_pipeline_config(self) -> dict: """ Saves the information on the composed pipeline configuration package into a YAML file. This YAML file describes the components and the data flow between them for the AI Inference Server. The file is created in the `destination` folder with name `pipeline_config.yml` """ version = self.save_version if self.save_version is not None else self.init_version metric_fields = [(component_name, field) for component_name, component in self.components.items() if isinstance(component, PythonComponent) for field in component.metrics.keys()] pipeline_inputs = [ i.__dict__(self.components[i.componentName].inputs) for i in self.inputs ] pipeline_outputs = [ {**o.__dict__(self.components[o.componentName].outputs), 'metric': False} for o in self.outputs ] pipeline_outputs += [{ 'name': field, 'type': 'String', 'metric': True, 'topic': f"/siemens/edge/aiinference/{self.name}/{version}/metrics/{component_name}/{field}", } for component_name, field in metric_fields] pipeline_dag = [{ 'source': f'Databus.{i.variableName}', 'target': f'{i.componentName}.{i.variableName}', } for i in self.inputs] pipeline_dag += [{ 'source': f"{wire['fromComponent']}.{wire['fromOutput']}", 'target': f"{wire['toComponent']}.{wire['toInput']}", } for wire in self.wiring.values()] pipeline_dag += [{ 'source': f'{o.componentName}.{o.variableName}', 'target': f'Databus.{o.variableName}', } for o in self.outputs] pipeline_dag += [{ 'source': f'{component_name}.{field}', 'target': f'Databus.{field}', } for component_name, field in metric_fields] config_yml_content = { 'fileFormatVersion': '1.2.0', 'dataFlowPipelineInfo': { 'author': self.author, 'createdOn': datetime.now(), 'dataFlowPipelineVersion': version, 'description': self.desc if self.desc else 'Created by AI SDK', 'projectName': self.name, 'packageId': str(self.package_id) }, 'dataFlowPipeline': { 'components': [component._to_dict() for component in self.components.values()], 'pipelineDag': pipeline_dag, 'pipelineInputs': pipeline_inputs, 'pipelineOutputs': pipeline_outputs, }, 'packageType': 'full' } if len(self.parameters) > 0: config_yml_content["dataFlowPipeline"]["pipelineParameters"] = [ param.__param_dict__() for param in self.parameters ] if self.cycle_time is not None: config_yml_content["dataFlowPipeline"]["cycle_time"] = self.cycle_time return config_yml_content ``` ### `save_pipeline_config(destination)` Saves the information about the composed pipeline configuration package into a YAML file. This YAML file describes the components and the data flow between them for AI Inference Server. The file will be created in the `destination` folder with name `pipeline_config.yml` Parameters: | Name | Type | Description | Default | | ------------- | ------------- | ---------------------------------- | ---------- | | `destination` | `path - like` | Path of the destination directory. | *required* | Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/pipeline.py` ```python def save_pipeline_config(self, destination): """ Saves the information about the composed pipeline configuration package into a YAML file. This YAML file describes the components and the data flow between them for AI Inference Server. The file will be created in the `destination` folder with name `pipeline_config.yml` Args: destination (path-like): Path of the `destination` directory. """ with open(Path(destination) / PIPELINE_CONFIG, "w") as f: yaml.dump(self.get_pipeline_config(), f) ``` ### `get_datalink_metadata()` The method generates metadata information based on available information. Returns: | Name | Type | Description | | ------ | ------ | ---------------------------------------------------------------------- | | `dict` | `dict` | Dictionary with the necessary information for the AI Inference Server. | Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/pipeline.py` ```python def get_datalink_metadata(self) -> dict: """ The method generates metadata information based on available information. Returns: dict: Dictionary with the necessary information for the AI Inference Server. """ timeshifting = { "id": None, "enabled": False, "periodicity": self.periodicity, "startingPoint": None, } if self.periodicity is not None: timeshifting["enabled"] = True timeshifting["startingPoint"] = 'First timestamp' if len(self.timeshift_reference) > 0: timeshifting["startingPoint"] = 'Signal reference' exported_metadata = { "fileFormatVersion": "1.0.0", "id": None, "version": None, "createdOn": datetime.now(), "updatedOn": datetime.now(), "timeShifting": timeshifting, "inputs": [ { 'name': input.variableName, 'mapping': None, 'timeShiftingReference': input.variableName in self.timeshift_reference, 'type': self.components[input.componentName].inputs[input.variableName]['type'] } for input in self.inputs ] } return exported_metadata ``` ### `save_datalink_metadata(destination)` Saves metadata for pipeline input variables. This method saves metadata for the AI Inference Server into a YAML file. This metadata determines how the AI Inference Server feeds input to the pipeline, especially inter-signal alignment. The file is created in the `destination` folder with the name `datalink_metadata.yml` Parameters: | Name | Type | Description | Default | | ------------- | ------------- | ---------------------------------- | ---------- | | `destination` | `path - like` | Path of the destination directory. | *required* | Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/pipeline.py` ```python def save_datalink_metadata(self, destination): """ Saves metadata for pipeline input variables. This method saves metadata for the AI Inference Server into a YAML file. This metadata determines how the AI Inference Server feeds input to the pipeline, especially inter-signal alignment. The file is created in the `destination` folder with the name `datalink_metadata.yml` Args: destination (path-like): Path of the destination directory. """ with open(Path(destination) / DATALINK_METADATA, "w") as f: yaml.dump(self.get_datalink_metadata(), f) ``` ### `save_telemetry_data(destination)` Save telemetry data to a specified destination. Parameters: | Name | Type | Description | Default | | ------------- | ------ | -------------------------------------------------- | ---------- | | `destination` | `Path` | The path where the telemetry data should be saved. | *required* | Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/pipeline.py` ```python def save_telemetry_data(self, destination: Path): """ Save telemetry data to a specified destination. Args: destination (Path): The path where the telemetry data should be saved. """ telemetry_path = destination / TELEMETRY_YAML telemetry_data = {} try: consent_allowed = is_telemetry_allowed() except RuntimeError: consent_allowed = "undecided" try: consent_data, _ = read_telemetry_consent_file() except BaseException: consent_data = {} telemetry_data["telemetry_consent"] = {} telemetry_data["telemetry_consent"][CONSENT_KEY] = consent_allowed telemetry_data["telemetry_consent"][CONSENT_KEY_SDK] = consent_data.get(CONSENT_KEY_SDK, 'unknown') telemetry_data["telemetry_consent"][CONSENT_KEY_TS] = consent_data.get(CONSENT_KEY_TS, 'unknown') telemetry_data["telemetry_consent"][CONSENT_KEY_METHOD] = consent_data.get(CONSENT_KEY_METHOD, get_consent_source()) telemetry_data["platform"] = {} telemetry_data["platform"]["os"] = platform.system() telemetry_data["platform"]["release"] = platform.release() telemetry_data["platform"]["python_version"] = platform.python_version() _logger.info(f"locals: {locals()}") telemetry_data["environment"] = {} env_info = "Jupyter" if any(k for k in locals() if k in ["__IPYTHON__", "get_ipython"]) else MSG_NOT_FOUND env_info = "GitLab CI/CD" if "GITLAB_CI" in os.environ else env_info env_info = "Azure DevOps Pipelines" if "TF_BUILD" in os.environ else env_info env_info = "GitHub Actions" if "GITHUB_ACTIONS" in os.environ else env_info telemetry_data["environment"]["info"] = env_info telemetry_data["industrial_ai"] = {} try: telemetry_data["industrial_ai"]["simaticai"] = importlib_metadata.version("simaticai") except importlib_metadata.PackageNotFoundError: telemetry_data["industrial_ai"]["simaticai"] = MSG_NOT_FOUND _logger.debug("simaticai package not found") telemetry_data["pipeline"] = {} telemetry_data["pipeline"]["python_versions"] = list(set(self.components[component].python_version for component in self.components if isinstance(self.components[component], PythonComponent))) telemetry_data["pipeline"]["file_extensions"] = list(set(f.suffix for f in Path(destination).rglob("*") if f.suffix not in ["", ".zip", ".yml", ".yaml", ".html"])) with open(telemetry_path, 'w') as telemetry_file: yaml.dump(telemetry_data, telemetry_file) ``` ### `validate(destination='.')` Validates whether the package configuration is compatible with the expected runtime environment. The method verifies: - If the package has at least one component - If all wires create connections between existing components and their variables - If metadata is defined and valid. - If a package with the same name already exists in the `destination` folder. In this case a warning message appears and the `save(..)` method overwrites the existing package. - If the package has multiple components and if they are using the same Python version Parameters: | Name | Type | Description | Default | | ------------- | ----- | --------------------------------------------------------- | ------- | | `destination` | `str` | Path of the expected destination folder. Defaults to ".". | `'.'` | Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/pipeline.py` ```python def validate(self, destination: str | Path = "."): """ Validates whether the package configuration is compatible with the expected runtime environment. The method verifies: - If the package has at least one component - If all wires create connections between existing components and their variables - If metadata is defined and valid. - If a package with the same name already exists in the `destination` folder. In this case a warning message appears and the `save(..)` method overwrites the existing package. - If the package has multiple components and if they are using the same Python version Args: destination (str, optional): Path of the expected destination folder. Defaults to ".". """ version = self.save_version if self.save_version is not None else self.init_version if len(self.components) < 1: raise AssertionError("The package must have at least one component.") for p_output in self.outputs: if self.components[p_output.componentName].batch.outputBatch: raise AssertionError(f"The component '{p_output.componentName}' has pipeline output defined with variable name '{p_output.variableName}'. \ None of component with pipeline output is allowed to provide batch output.") for wire_hash in self.wiring.copy(): wire = self.wiring[wire_hash] self._check_wiring(wire, wire_hash) pipeline_inputs = [p_input.variableName for p_input in self.inputs] pipeline_outputs = [p_output.variableName for p_output in self.outputs] if any(variable in pipeline_outputs for variable in pipeline_inputs): conflicts = set(pipeline_inputs).intersection(set(pipeline_outputs)) raise AssertionError(f"Pipeline input and output variables must be unique. Conflicting variables: {conflicts}") self._check_timeshifting() self._check_cycle_time() self._check_supported_types_for_connections() package_path = Path(destination) / f"{self.name}_{version}".replace(" ", "-") if package_path.is_dir(): _logger.warning(f"Target folder ({package_path}) already exists! Unless changing the package name the package could be invalid and your files will be overwritten!") python_versions = set() for component in self.components: self.components[component].validate() if isinstance(self.components[component], PythonComponent): python_versions.add(self.components[component].python_version) if (1 < len(python_versions)): _logger.warning("The use of multiple python version in a single pipeline is not recommended. We recommend using only one of the supported versions, which are Python 3.11 or 3.12.") _logger.info(f"Package '{self.name}' is valid and ready to save.") ``` ### `save(destination='.', package_id=None, version=None)` @Deprecated, reason: only edge package generation will be supported in the future. Use export instead. Saves the assembled package in a zip format. The name of the file is defined as `{package_name}_{package_version}.zip`. If a file with such a name already exists in the `destination` folder, it gets overwritten and a warning message appears. The package is also available as a subfolder on the destination path with the name `{package_name}_{package_version}`. If the assembled content does not meet the expected one, this content can be changed and simply packed into a zip file. The package contains files and folders in the following structure: - Package folder with name `{package_name}_{package_version}` - `datalink-metadata.yml` - `pipeline-config.yml` - Component folder with name `{component_name}` When the component is a `PythonComponent`, this folder contains: - `requirements.txt` - Entrypoint script defined by the entrypoint of the component - Extra files as added to the specified folders - Source folder with name `src` with necessary python scripts If a package ID is specified, and a package with the same ID and version is already present in the `destination` folder, an error is raised. Parameters: | Name | Type | Description | Default | | ------------- | ------ | ---------------------------------------------------------- | ------- | | `destination` | `str` | Target directory for saving the package. Defaults to ".". | `'.'` | | `package_id` | `UUID` | The optional package ID. If None, a new UUID is generated. | `None` | Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/pipeline.py` ```python def save(self, destination: str | Path = ".", package_id: Optional[uuid.UUID] = None, version: Optional[str] = None) -> Path: """ @Deprecated, reason: only edge package generation will be supported in the future. Use export instead. Saves the assembled package in a zip format. The name of the file is defined as `{package_name}_{package_version}.zip`. If a file with such a name already exists in the `destination` folder, it gets overwritten and a warning message appears. The package is also available as a subfolder on the destination path with the name `{package_name}_{package_version}`. If the assembled content does not meet the expected one, this content can be changed and simply packed into a zip file. The package contains files and folders in the following structure: - Package folder with name `{package_name}_{package_version}` - `datalink-metadata.yml` - `pipeline-config.yml` - Component folder with name `{component_name}` When the component is a `PythonComponent`, this folder contains: - `requirements.txt` - Entrypoint script defined by the entrypoint of the component - Extra files as added to the specified folders - Source folder with name `src` with necessary python scripts If a package ID is specified, and a package with the same ID and version is already present in the `destination` folder, an error is raised. Args: destination (str, optional): Target directory for saving the package. Defaults to ".". package_id (UUID): The optional package ID. If None, a new UUID is generated. """ try: # checking whether the user has made a statement about telemetry collection is_telemetry_allowed() except RuntimeError as consent_is_undecided: raise consent_is_undecided self._set_save_version_and_package_id(Path(destination), package_id, version) destination = Path(destination) self.validate(destination) name = self.name.replace(" ", "-") package_name = f"{name}_{self.save_version}" destination = destination / package_name destination.mkdir(parents=True, exist_ok=True) for component in self.components: self.components[component].save(destination, False) if isinstance(self.components[component], PythonComponent): self.report_writer.add_direct_dependencies(self.components[component].name, self.components[component].python_dependencies.dependencies) self.save_datalink_metadata(destination) self.save_pipeline_config(destination) p_page.save_readme_html(self, destination) self.save_telemetry_data_if_consented(destination) zip_destination = shutil.make_archive( base_name=str(destination.parent / package_name), format='zip', root_dir=destination.parent, base_dir=package_name, verbose=True, logger=_logger) return Path(zip_destination) ``` ### `export(destination='.', package_id=None, version=None)` Export a runnable pipeline package. Parameters: | Name | Type | Description | Default | | ------------- | ------ | -------------------------------------------------------------------- | ------- | | `destination` | `str` | optional target directory for saving the package. Defaults to ".". | `'.'` | | `package_id` | `UUID` | optional package ID. If None, a new UUID is generated. | `None` | | `version` | `str` | optional version. If None, an automatic version number is generated. | `None` | Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/pipeline.py` ```python def export(self, destination: str | Path = ".", package_id: Optional[uuid.UUID] = None, version: Optional[str] = None) -> Path: """ Export a runnable pipeline package. Args: destination (str): optional target directory for saving the package. Defaults to ".". package_id (UUID): optional package ID. If None, a new UUID is generated. version (str): optional version. If None, an automatic version number is generated. """ try: # checking whether the user has made a statement about telemetry collection is_telemetry_allowed() except RuntimeError as consent_is_undecided: raise consent_is_undecided destination = Path(destination) config_package = None try: config_package = self.save(destination, package_id, version) runtime_package = convert_package(config_package, self.report_writer) return runtime_package finally: if config_package is not None: Path(config_package).unlink(missing_ok=True) ``` ### `add_secret_parameter(name, desc=None)` Adds a secret parameter to the pipeline configuration. Parameters: | Name | Type | Description | Default | | ------ | ----- | ------------------------------------------------------------------------------------------------------------------------------ | ---------- | | `name` | `str` | Name of the secret parameter. Must contain only letters, digits, underscores, and hyphens. Cannot start with \__AI_IS_ prefix. | *required* | | `desc` | `str` | Description of the secret parameter. Must be a string between 3 and 60 characters if provided. | `None` | Raises ValueError if the conditions for name or description are not met. Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/pipeline.py` ```python def add_secret_parameter(self, name: str, desc: str | None = None): """ Adds a secret parameter to the pipeline configuration. Args: name (str): Name of the secret parameter. Must contain only letters, digits, underscores, and hyphens. Cannot start with `__AI_IS_` prefix. desc (str, optional): Description of the secret parameter. Must be a string between 3 and 60 characters if provided. Raises ValueError if the conditions for name or description are not met. """ self._add_parameter(SecretPipelineParameter(name=name, description=desc)) ``` ### `add_parameter(name, default_value, type_name='String', topic_based=False, desc=None)` Adds a parameter to the pipeline configuration, which alters the behavior of the pipeline. The parameter's default value and its properties are saved in the pipeline configuration and the value of the parameter can later be changed on AI Inference Server. Parameters: | Name | Type | Description | Default | | --------------- | ------ | ----------------------------------------------------------- | ---------- | | `name` | `str` | Name of the parameter | *required* | | `desc` | `str` | Description of the parameter (optional) | `None` | | `type_name` | `str` | Data type of the parameter. Defaults to "String". | `'String'` | | `default_value` | `str` | Default value of the parameter | *required* | | `topic_based` | `bool` | If true, the parameter can be updated from a message queue. | `False` | ```text ValueError: When: - the default value of the parameter is not of the specified data type (`type_name`) or - the specified data type itself is not an allowed data type (not a part of `parameter_types` dict) or - the specified data type is not given in the right format or - the type of the given `topic_based` parameter is not `bool`. - the name of the parameter starts with `__AI_IS_` prefix. These are reserved parameters by AI Inference Server ``` Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/pipeline.py` ```python def add_parameter(self, name: str, default_value, type_name: str = "String", topic_based: bool = False, desc: str = None): """ Adds a parameter to the pipeline configuration, which alters the behavior of the pipeline. The parameter's default value and its properties are saved in the pipeline configuration and the value of the parameter can later be changed on AI Inference Server. Args: name (str): Name of the parameter desc (str): Description of the parameter (optional) type_name (str, optional): Data type of the parameter. Defaults to "String". default_value (str): Default value of the parameter topic_based (bool, optional): If true, the parameter can be updated from a message queue. Raises: ValueError: When: - the default value of the parameter is not of the specified data type (`type_name`) or - the specified data type itself is not an allowed data type (not a part of `parameter_types` dict) or - the specified data type is not given in the right format or - the type of the given `topic_based` parameter is not `bool`. - the name of the parameter starts with `__AI_IS_` prefix. These are reserved parameters by AI Inference Server """ self._add_parameter(PipelineParameter( name=name, defaultValue=default_value, dtype=type_name, topicBased=topic_based, description=desc)) ``` ## `convert_package(zip_path, report_writer=None)` @Deprecated, reason: only edge package generation will be supported in the future. Use Pipeline.export(...) instead. Create an Edge Configuration Package from a given Pipeline Configuration Package. If the input zip file is `{path}/{name}_{version}.zip`, the output file will be created as `{path}/{name}-edge_{version}.zip`. Please make sure that the given zip file comes from a trusted source! If a file with such a name already exists, it is overwritten. First, this method verifies that the requirements identified by name and version are either included in `PythonPackages.zip` or available on pypi.org for the target platform. Currently, the supported edge devices run Linux on 64-bit x86 architecture, so the accepted Python libraries are restricted to the platform independent ones and packages built for 'x86_64' platforms. AI Inference Server also provides a Python 3.11 and 3.12 runtime environment, so the supported Python libraries are restricted to Python 3.11 and 3.12 compatible packages. If for the target platform the required dependency is not available on pypi.org and not present in `PythonPackages.zip`, it will log the problem at ERROR level. Then it downloads all dependencies (either direct or transitive), and creates a new zip file, which is validated against the AI Inference Server's schema. This functionality requires pip with version of 21.3.1 or greater. This method can be used from the command line too. Example usage: ```text python -m simaticai convert_package ``` Parameters: | Name | Type | Description | Default | | --------------- | -------------- | --------------------------------------------------------------------------- | ---------- | | `zip_path` | `path - like` | path to the pipeline configuration package zip file. | *required* | | `report_writer` | `ReportWriter` | a ReportWriter object to write the report for a pipeline. Defaults to None. | `None` | Returns: | Type | Description | | ------ | ---------------------------------------------- | | `Path` | os.PathLike: The path of the created zip file. | Raises: | Type | Description | | ------------------------- | ----------------------------------------------------------- | | `PipelineValidationError` | If the validation fails. See the logger output for details. | Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/pipeline.py` ````python def convert_package(zip_path: Union[str, os.PathLike], report_writer: Optional[PipelineReportWriter] = None) -> Path: """ @Deprecated, reason: only edge package generation will be supported in the future. Use Pipeline.export(...) instead. Create an Edge Configuration Package from a given Pipeline Configuration Package. If the input zip file is `{path}/{name}_{version}.zip`, the output file will be created as `{path}/{name}-edge_{version}.zip`. Please make sure that the given zip file comes from a trusted source! If a file with such a name already exists, it is overwritten. First, this method verifies that the requirements identified by name and version are either included in `PythonPackages.zip` or available on pypi.org for the target platform. Currently, the supported edge devices run Linux on 64-bit x86 architecture, so the accepted Python libraries are restricted to the platform independent ones and packages built for 'x86_64' platforms. AI Inference Server also provides a Python 3.11 and 3.12 runtime environment, so the supported Python libraries are restricted to Python 3.11 and 3.12 compatible packages. If for the target platform the required dependency is not available on pypi.org and not present in `PythonPackages.zip`, it will log the problem at ERROR level. Then it downloads all dependencies (either direct or transitive), and creates a new zip file, which is validated against the AI Inference Server's schema. This functionality requires pip with version of 21.3.1 or greater. This method can be used from the command line too. Example usage: ``` python -m simaticai convert_package ``` Args: zip_path (path-like): path to the pipeline configuration package zip file. report_writer (ReportWriter, optional): a ReportWriter object to write the report for a pipeline. Defaults to None. Returns: os.PathLike: The path of the created zip file. Exceptions: PipelineValidationError: If the validation fails. See the logger output for details. """ try: # checking whether the user has made a statement about telemetry collection is_telemetry_allowed() except RuntimeError as consent_is_undecided: raise consent_is_undecided zip_path = Path(zip_path) if zip_path.stem.find('_') < 0: raise AssertionError("The input zip file name must contain an underscore character.") with tempfiles.OpenZipInTemp(zip_path) as zip_dir: top_level_items = list(zip_dir.iterdir()) if len(top_level_items) != 1: raise AssertionError("The Pipeline Configuration Package must contain a single top level directory.") package_dir = zip_dir / top_level_items[0] runtime_dir = zip_dir / "edge_config_package" runtime_dir.mkdir(parents=True, exist_ok=True) config = yaml_helper.read_yaml(package_dir / PIPELINE_CONFIG) _validate_with_schema("input pipeline_config.yml", config, "pipeline.schema.json") runtime_config = _generate_runtime_config(config) if report_writer is not None: report_writer.set_path(Path(zip_path.parent / f"{zip_path.stem}_package_report.md")) report_writer.set_pipeline_config(config) for component in config['dataFlowPipeline']['components']: source_dir = package_dir / component["name"] if component["runtime"]["type"] == "python": python_version = component['runtime']['version'] try: python_version_validator(python_version) except ValueError as error: raise AssertionError(error) pip_option_no_deps = component["runtime"].get("installTransitiveDependencies", True) is False if pip_option_no_deps: _logger.warning(f"Component `{component['name']}` is requested to be installed without transitive dependencies. " "This may lead to runtime errors if the dependencies are not handled manually.") dependency_set = _package_component_dependencies(source_dir, python_version, pip_option_no_deps) if report_writer is not None: report_writer.add_full_dependency_set(component_name=component["name"], dependency_set=dependency_set) runtime_config["runtimeConfiguration"]["components"].append({ "name": component["name"], "device": "IED1", "targetRuntime": "Python", }) if component["runtime"]["type"] == "gpuruntime": runtime_config["runtimeConfiguration"]["components"].append({ "name": component["name"], "device": "IED1", "targetRuntime": "gpuruntime", }) _package_component(source_dir, runtime_dir / 'components' / f"{component['name']}_{component['version']}") if report_writer is not None: report_writer.write_report() _logger.info(f"Report on {zip_path.stem} is saved to {zip_path.parent}.") shutil.copy(str(package_dir / PIPELINE_CONFIG), str(runtime_dir / PIPELINE_CONFIG)) datalink_metadata_yaml = package_dir / DATALINK_METADATA if datalink_metadata_yaml.is_file(): shutil.copy(str(datalink_metadata_yaml), runtime_dir / DATALINK_METADATA) _validate_with_schema(f"generated {RUNTIME_CONFIG}", runtime_config, "runtime.schema.json") with open(runtime_dir / RUNTIME_CONFIG, "w", encoding="utf8") as file: yaml.dump(runtime_config, file) readme_html = package_dir / README_HTML if readme_html.exists(): (runtime_dir / README_HTML).write_text(readme_html.read_text()) telemetry_yaml = package_dir / TELEMETRY_YAML if telemetry_yaml.exists(): (runtime_dir / TELEMETRY_YAML).write_text(telemetry_yaml.read_text()) edge_package_path = Path(shutil.make_archive( # replacing the last occurrence of "_" with "-edge_". base_name=str(PurePath(zip_path).parent / "-edge_".join(zip_path.stem.rsplit("_", 1))), format='zip', root_dir=runtime_dir, verbose=True, logger=_logger)) package_size = edge_package_path.stat().st_size # zipped package size in bytes if package_size > PACKAGE_SIZE_LIMIT: package_size_GB = "{:.2f}".format(package_size / 1000 / 1000 / 1000) package_size_limit_GB = "{:.2f}".format(PACKAGE_SIZE_LIMIT / 1000 / 1000 / 1000) error_msg = f"Pipeline package size {package_size} bytes ({package_size_GB} GB) exceeds the limit of " \ f"{PACKAGE_SIZE_LIMIT} bytes ({package_size_limit_GB} GB). " \ "Please remove unnecessary files and dependencies and try again." _logger.error(error_msg) raise RuntimeError(error_msg) sha256_hash = calc_sha(edge_package_path) sha_format = f"{sha256_hash} {edge_package_path.name}" edge_package_path.with_suffix('.sha256').write_text(sha_format) return edge_package_path ```` ## `PythonComponent` Bases: `Component` A pipeline component implemented using Python scripts and libraries. A `PythonComponent` wraps Python code resource files such as saved models into a structured folder, which can be added to a pipeline configuration package. For a comprehensive overview on how to wrap ML models into Python components, we recommend you refer to the AI SDK User Manual, especially the guideline for writing pipeline components. We also recommend you study the example Python components in the E2E Tutorials for the AI SDK. A new `PythonComponent` is empty. Parameters: | Name | Type | Description | Default | | ---------------- | ----- | ------------------------------------------------------------------------------------------------------------------------------ | ------------- | | `name` | `str` | Component name. (default: inference) | `'inference'` | | `desc` | `str` | Component description (optional) | `''` | | `version` | `str` | Component version. (default: 0.0.1) | `'0.0.1'` | | `python_version` | `str` | Python version on the target AI Inference Server. At the moment of writing, the current version supports Python 3.11 and 3.12. | `'3.12'` | Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/python_component.py` ````python class PythonComponent(Component): """ A pipeline component implemented using Python scripts and libraries. A `PythonComponent` wraps Python code resource files such as saved models into a structured folder, which can be added to a pipeline configuration package. For a comprehensive overview on how to wrap ML models into Python components, we recommend you refer to the AI SDK User Manual, especially the guideline for writing pipeline components. We also recommend you study the example Python components in the E2E Tutorials for the AI SDK. A new `PythonComponent` is empty. Args: name (str): Component name. (default: inference) desc (str): Component description (optional) version (str): Component version. (default: 0.0.1) python_version (str): Python version on the target AI Inference Server. At the moment of writing, the current version supports Python 3.11 and 3.12. """ def __init__(self, name: str = "inference", version: str = "0.0.1", python_version: str = '3.12', desc: str = ""): """ Creates a new, empty Python component. Args: name (str): Component name. (default: inference) desc (str): Component description (optional) version (str): Component version. (default: 0.0.1) python_version (str): Python version on the target AI Inference Server. At the moment of writing, AI Inference Server supports Python 3.11 and 3.12. """ super().__init__(name=name, desc=desc) try: python_version_validator(python_version) except ValueError as error: raise AssertionError(error) self.python_version = str(python_version) self.version = version self.metrics = {} self.entrypoint: Path | None = None self.resources = {} self.python_dependencies = PythonDependencies(python_version) self._replicas = 1 self.is_valid = False self.no_deps = False def __repr__(self) -> str: text = super().__repr__() if len(self.metrics) > 0: text += "\nMetrics:\n" for name, metric in self.metrics.items(): text += f"< {name}{': ' + metric['desc'] if metric.get('desc') is not None else ''}\n" if len(self.resources): text += "\nResources:\n" for path, base in self.resources.items(): text += f" {base}/{path.name}\n".replace('./', '') if self.entrypoint is not None: text += f"Entrypoint: {self.entrypoint}\n" return text def set_entrypoint(self, entrypoint: str): """ Sets the entrypoint module for the component. The entrypoint is the Python code which is responsible for receiving the input data and producing a structured response with the output for the AI Inference Server. The script should consume a JSON string and produce another. See the short example below. The file will be copied into the root directory of the component on the AI Inference Server, so every file reference should be aligned. The example code below shows a basic structure of the entrypoint Python code. ```python import json import sys from pathlib import Path # by adding the parent folder of your modules to system path makes them available for relative import sys.path.insert(0, str(Path('./src').resolve())) from my_module import processor # then the processor module can be imported def run(data: str): input_data = json.loads(data) # incoming JSON string is loaded as a dictionary result = processor.process_data(input_data) # the process_data can be called to process the incoming data # the code below creates the formatted output for the AI Inference Server if result is None: answer = {"ready": False, "output": None} else: answer = {"ready": True, "output": json.dumps(result)} return answer ``` Args: entrypoint (str): Name of the new entrypoint script to be copied """ self.is_valid = False if not any(key.name for key, value in self.resources.items() if key.name == entrypoint and value == '.'): raise AssertionError("Entrypoint must be added as resource to the root directory before setting up as entrypoint.") self.entrypoint = Path(entrypoint) def add_resources(self, base_dir: os.PathLike, resources: os.PathLike | list): """ Adds files to a component. To make your file resources available on the AI Inference Server you need to add them to the package resources. These resources can be Python or config files, serialized ML models or reference data. They are then available on path {component_root}/{resources} in the runtime environment. When saving the package they will be copied from {base_dir}/{resources} into the package. Files in '__pycache__' folders will be excluded. Until version 2.3.0 of AI SDK hidden files and folders (starting with '.') are also excluded. Args: base_dir (path-like): Root folder of your code from which the resources are referred resources (os.PathLike or List): A single path or list of relative paths to resource files """ self.is_valid = False base_dir = Path(base_dir).resolve().absolute() if not base_dir.is_dir(): raise AssertionError(f"Parameter 'base_dir' must be a directory and available in path {base_dir}.") resources = resources if type(resources) is list else [resources] for resource in resources: self._add_resource(base_dir, resource) def _add_resource(self, base_dir: Path, resource: os.PathLike): self.is_valid = False if Path(resource).is_absolute() or '..' in resource: raise AssertionError("The resource path must be relative and cannot contain '/../' elements.") resource_path = base_dir / resource if resource_path.is_file(): self._add_resource_file(base_dir, resource_path) return if resource_path.is_dir(): for glob_path in resource_path.rglob("*"): if glob_path.is_file(): self._add_resource_file(base_dir, glob_path) return raise AssertionError(f"Specified resource is not a file or directory: '{resource}'") def _add_resource_file(self, base_dir: Path, resource_path: Path): self.is_valid = False for parent in resource_path.parents: if parent.name == '__pycache__': return if resource_path in self.resources.keys(): _logger.warning(f"Resource '{resource_path}' is already added to target directory '{self.resources[resource_path]}'") return self.resources[resource_path] = f"{resource_path.parent.relative_to(base_dir)}" def add_dependencies(self, packages: list): """ Adds required dependencies for the Python code. The list must contain the name of the Python packages or tuples in the form of (name, version) which are required to execute the component on AI Inference Server. The method will search for the packages for the target platform and collect their transitive dependencies as well. Packages that are distributed only in source format can be added too, but only if they are pure Python packages. Args: packages (list): Can be a list of strings (name) or a list of tuples (name, version) of the required packages for component execution """ self.is_valid = False self.python_dependencies.add_dependencies(packages) def set_requirements(self, requirements_path: os.PathLike, no_deps: bool = False): """ Reads the defined dependencies from the given `requirements.txt` file and creates a new dependency list. Previously added dependencies will be cleared. The file format must follow Python's requirements file format defined in PEP 508. It can contain URLs to additional repositories in the form of `--extra-index-url=my.repo.example.com`. Args: requirements_path (str): Path of the given `requirements.txt` file no_deps (bool): If set to True, dependencies will be downloaded/installed without dependencies. Default False. """ self.is_valid = False self.python_dependencies.set_requirements(requirements_path) if no_deps: self.no_deps = True _logger.warning("WARNING! Requiring packages without their transitive dependencies is only supported on AI Inference Server version >=2.6.0") _logger.warning("WARNING! Requiring packages without their transitive dependencies may lead to missing dependencies and runtime errors. Use with caution!") def set_pyproject_toml(self, pyproject_path: os.PathLike): """ Reads the defined dependencies from the given `pyproject.toml` file and adds it to the requirements list. Only the dependencies defined in the `[project]` section will be added to the component. Args: pyproject_path (str): Path of the given `pyproject.toml` file """ self.is_valid = False self.python_dependencies.set_pyproject_toml(pyproject_path) def add_python_packages(self, path: str) -> None: """ Adds Python package(s) to the `PythonPackages.zip` file of the component. The `path` parameter can refer to either a `whl`, a `zip` or a `tar.gz` file. Zip files can be either a source distribution package or a collection of Python packages. Only pure Python source distributions are allowed. The dependency list of the component will be extended with the files added here, so that they will also get installed on the AI Inference Server. The method uses the `tempfile.tempdir` folder, so make sure that the folder is writeable. The wheel files must fulfill the requirements of the targeted device environment (e.g., the Python version must match the supported Python version of the targeted AI Inference Server, and the platform should be one of the supported ones too). Args: path (str): Path of the distribution file Examples: `component.add_python_packages('../resources/my_package-0.0.1-py3-none-any.wheel')` adds the wheel file to `PythonPackages.zip` and adds dictionary item `component.dependencies['my_package'] = '0.0.1'` `component.add_python_packages('../resources/inference-wheels.zip')` adds all the wheel files in the zip to `PythonPackages.zip` and `component.dependencies` """ self.is_valid = False self.python_dependencies.add_python_packages(path) def set_parallel_steps(self, replicas): """ Sets the number of parallel executors. This method configures how many instances of the component can be executed at the same time. The component must be suitable for parallel execution. The inputs arriving to the component will be processed by different instances in parallel, and these instances do not share their state (e.g. variables). Every instance is initialized separately and receives only a fraction of the inputs. AI Inference Server supports at most 8 parallel instances.``` Args: replicas (int): Number of parallel executors. Default is 1. Raises: ValueError: if the given argument is not a positive integer. """ self.is_valid = False if (not isinstance(replicas, int)) or replicas < 1: raise ValueError("Replica count must be a positive integer.") if 8 < replicas: _logger.warning("The current maximum of parallel executors is 8.") self._replicas = replicas def add_metric(self, name: str, desc: str | None = None): """ Adds a metric that will be automatically used as a pipeline output. Args: name (str): Name of the metric. desc (str): Description of the metric. (optional) """ if "_" not in name: raise AssertionError("The metric name must contain at least one underscore") if self.metrics is None: self.metrics = {} if name in self.metrics: raise AssertionError(f"Metric '{name}' already exists") self.metrics[name] = {} if desc is not None: self.metrics[name]['desc'] = desc def delete_metric(self, name: str): """ Remove a previously added metric. Args: name (str): Name of the metric to be deleted. """ if name not in self.metrics: raise AssertionError(f"Component '{self.name}' has no metric '{name}'") self.metrics.pop(name) def _to_dict(self): runtime = { 'type': 'python', 'version': self.python_version, } if self.no_deps: runtime['installTransitiveDependencies'] = False component_dict = { **super()._to_dict(), 'version': self.version, 'entrypoint': f"./{self.entrypoint.name}", 'hwType': 'CPU', 'runtime': runtime, 'replicas': self._replicas } component_dict["outputType"] += [{ 'name': name, 'type': 'String', 'metric': True, } for name in self.metrics.keys()] return component_dict def enable_dependency_optimization(self): """ Allows changing repository URLs to optimize the package size Allows the replacement of the `--index-url` argument during `pip download` to download CPU runtime optimized dependencies only. Enabling this optimization, the present `--index-url` will be prepended to the `--extra-index-url` list, and the Pytorch CPU only repository will be set as the `--index-url`. A warning message will be printed if the repository URL modification was necessary. Some dependencies have both CPU and GPU runtime versions, pytorch for example, but a `PythonComponent` can only run on CPU, so packaging the additional GPU runtime dependencies just enlarges the package size. If you want to run your model on GPU, convert it to an `ONNX` model and use it within a `GPURuntimeComponent`. """ self.python_dependencies.enable_dependency_optimization() def disable_dependency_optimization(self): """ Disables any modification to repository URLs Disables the replacement of the `--index-url` argument during `pip download`. This way all `--index-url` or `--extra-index-url` arguments will be preserved if they were present in the requirements.txt file. Some dependencies have both CPU and GPU runtime versions, pytorch for example, but a `PythonComponent` can only run on CPU, so packaging the additional GPU runtime dependencies just enlarges the package size. A warning message will be printed about the package size if this optimization is disabled and the dependency list contains GPU optimized dependencies. Disabling this optimization will not allow the component to run on GPU. If you want to run your model on GPU, convert it to an `ONNX` model and use it within a `GPURuntimeComponent`. """ self.python_dependencies.disable_dependency_optimization() def validate(self): """ Validates that the component is ready to be serialized and packaged as part of a pipeline. """ if not self.is_valid: if self.entrypoint is None: raise AssertionError("Entrypoint must be defined") if not any(key.name for key, value in self.resources.items() if key.name == self.entrypoint.name and value == '.'): raise AssertionError("Entrypoint must be added as resource to the root directory before setting up as entrypoint.") if len(self.python_dependencies.dependencies) < 1: _logger.warning(f"WARNING! There are no dependencies defined for component '{self.name}'. Please make sure that all necessary dependencies have been added.") self.python_dependencies.validate() self.is_valid = True _logger.info(f"Component '{self.name}' is valid and ready to use.") def save(self, destination, validate=True): """ Saves the component to a folder structure, so it can be used as part of a pipeline configuration package. Validation can be skipped by setting parameter `validate` to False. This is useful when the component is already validated and only intended to be saved. The component folder contains the following: - `requirements.txt` with a list of Python dependencies - Entry point script defined by the `entrypoint` attribute of the component - Extra files as added to the specified folders - `PythonPackages.zip` with the wheel binaries for the environment to be installed Args: destination (path-like): Target directory to which the component will be saved. validate (bool): With value True, triggers component validation. Defaults to True. """ if validate: self.validate() folder_path = Path(destination) / self.name folder_path.mkdir(parents=True, exist_ok=True) for file_path in self.resources: dir_path = folder_path / self.resources[file_path] os.makedirs(dir_path, exist_ok=True) shutil.copy(file_path, dir_path / file_path.name) self.python_dependencies.save(folder_path) ```` ### `__init__(name='inference', version='0.0.1', python_version='3.12', desc='')` Creates a new, empty Python component. Parameters: | Name | Type | Description | Default | | ---------------- | ----- | ------------------------------------------------------------------------------------------------------------------------------ | ------------- | | `name` | `str` | Component name. (default: inference) | `'inference'` | | `desc` | `str` | Component description (optional) | `''` | | `version` | `str` | Component version. (default: 0.0.1) | `'0.0.1'` | | `python_version` | `str` | Python version on the target AI Inference Server. At the moment of writing, AI Inference Server supports Python 3.11 and 3.12. | `'3.12'` | Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/python_component.py` ```python def __init__(self, name: str = "inference", version: str = "0.0.1", python_version: str = '3.12', desc: str = ""): """ Creates a new, empty Python component. Args: name (str): Component name. (default: inference) desc (str): Component description (optional) version (str): Component version. (default: 0.0.1) python_version (str): Python version on the target AI Inference Server. At the moment of writing, AI Inference Server supports Python 3.11 and 3.12. """ super().__init__(name=name, desc=desc) try: python_version_validator(python_version) except ValueError as error: raise AssertionError(error) self.python_version = str(python_version) self.version = version self.metrics = {} self.entrypoint: Path | None = None self.resources = {} self.python_dependencies = PythonDependencies(python_version) self._replicas = 1 self.is_valid = False self.no_deps = False ``` ### `set_entrypoint(entrypoint)` Sets the entrypoint module for the component. The entrypoint is the Python code which is responsible for receiving the input data and producing a structured response with the output for the AI Inference Server. The script should consume a JSON string and produce another. See the short example below. The file will be copied into the root directory of the component on the AI Inference Server, so every file reference should be aligned. The example code below shows a basic structure of the entrypoint Python code. ```python import json import sys from pathlib import Path # by adding the parent folder of your modules to system path makes them available for relative import sys.path.insert(0, str(Path('./src').resolve())) from my_module import processor # then the processor module can be imported def run(data: str): input_data = json.loads(data) # incoming JSON string is loaded as a dictionary result = processor.process_data(input_data) # the process_data can be called to process the incoming data # the code below creates the formatted output for the AI Inference Server if result is None: answer = {"ready": False, "output": None} else: answer = {"ready": True, "output": json.dumps(result)} return answer ``` Parameters: | Name | Type | Description | Default | | ------------ | ----- | ---------------------------------------------- | ---------- | | `entrypoint` | `str` | Name of the new entrypoint script to be copied | *required* | Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/python_component.py` ````python def set_entrypoint(self, entrypoint: str): """ Sets the entrypoint module for the component. The entrypoint is the Python code which is responsible for receiving the input data and producing a structured response with the output for the AI Inference Server. The script should consume a JSON string and produce another. See the short example below. The file will be copied into the root directory of the component on the AI Inference Server, so every file reference should be aligned. The example code below shows a basic structure of the entrypoint Python code. ```python import json import sys from pathlib import Path # by adding the parent folder of your modules to system path makes them available for relative import sys.path.insert(0, str(Path('./src').resolve())) from my_module import processor # then the processor module can be imported def run(data: str): input_data = json.loads(data) # incoming JSON string is loaded as a dictionary result = processor.process_data(input_data) # the process_data can be called to process the incoming data # the code below creates the formatted output for the AI Inference Server if result is None: answer = {"ready": False, "output": None} else: answer = {"ready": True, "output": json.dumps(result)} return answer ``` Args: entrypoint (str): Name of the new entrypoint script to be copied """ self.is_valid = False if not any(key.name for key, value in self.resources.items() if key.name == entrypoint and value == '.'): raise AssertionError("Entrypoint must be added as resource to the root directory before setting up as entrypoint.") self.entrypoint = Path(entrypoint) ```` ### `add_resources(base_dir, resources)` Adds files to a component. To make your file resources available on the AI Inference Server you need to add them to the package resources. These resources can be Python or config files, serialized ML models or reference data. They are then available on path {component_root}/{resources} in the runtime environment. When saving the package they will be copied from {base_dir}/{resources} into the package. Files in '**pycache**' folders will be excluded. Until version 2.3.0 of AI SDK hidden files and folders (starting with '.') are also excluded. Parameters: | Name | Type | Description | Default | | ----------- | ------------------ | -------------------------------------------------------------- | ---------- | | `base_dir` | `path - like` | Root folder of your code from which the resources are referred | *required* | | `resources` | `PathLike or List` | A single path or list of relative paths to resource files | *required* | Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/python_component.py` ```python def add_resources(self, base_dir: os.PathLike, resources: os.PathLike | list): """ Adds files to a component. To make your file resources available on the AI Inference Server you need to add them to the package resources. These resources can be Python or config files, serialized ML models or reference data. They are then available on path {component_root}/{resources} in the runtime environment. When saving the package they will be copied from {base_dir}/{resources} into the package. Files in '__pycache__' folders will be excluded. Until version 2.3.0 of AI SDK hidden files and folders (starting with '.') are also excluded. Args: base_dir (path-like): Root folder of your code from which the resources are referred resources (os.PathLike or List): A single path or list of relative paths to resource files """ self.is_valid = False base_dir = Path(base_dir).resolve().absolute() if not base_dir.is_dir(): raise AssertionError(f"Parameter 'base_dir' must be a directory and available in path {base_dir}.") resources = resources if type(resources) is list else [resources] for resource in resources: self._add_resource(base_dir, resource) ``` ### `add_dependencies(packages)` Adds required dependencies for the Python code. The list must contain the name of the Python packages or tuples in the form of (name, version) which are required to execute the component on AI Inference Server. The method will search for the packages for the target platform and collect their transitive dependencies as well. Packages that are distributed only in source format can be added too, but only if they are pure Python packages. Parameters: | Name | Type | Description | Default | | ---------- | ------ | -------------------------------------------------------------------------------------------------------------------- | ---------- | | `packages` | `list` | Can be a list of strings (name) or a list of tuples (name, version) of the required packages for component execution | *required* | Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/python_component.py` ```python def add_dependencies(self, packages: list): """ Adds required dependencies for the Python code. The list must contain the name of the Python packages or tuples in the form of (name, version) which are required to execute the component on AI Inference Server. The method will search for the packages for the target platform and collect their transitive dependencies as well. Packages that are distributed only in source format can be added too, but only if they are pure Python packages. Args: packages (list): Can be a list of strings (name) or a list of tuples (name, version) of the required packages for component execution """ self.is_valid = False self.python_dependencies.add_dependencies(packages) ``` ### `set_requirements(requirements_path, no_deps=False)` Reads the defined dependencies from the given `requirements.txt` file and creates a new dependency list. Previously added dependencies will be cleared. The file format must follow Python's requirements file format defined in PEP 508. It can contain URLs to additional repositories in the form of `--extra-index-url=my.repo.example.com`. Parameters: | Name | Type | Description | Default | | ------------------- | ------ | ---------------------------------------------------------------------------------------------- | ---------- | | `requirements_path` | `str` | Path of the given requirements.txt file | *required* | | `no_deps` | `bool` | If set to True, dependencies will be downloaded/installed without dependencies. Default False. | `False` | Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/python_component.py` ```python def set_requirements(self, requirements_path: os.PathLike, no_deps: bool = False): """ Reads the defined dependencies from the given `requirements.txt` file and creates a new dependency list. Previously added dependencies will be cleared. The file format must follow Python's requirements file format defined in PEP 508. It can contain URLs to additional repositories in the form of `--extra-index-url=my.repo.example.com`. Args: requirements_path (str): Path of the given `requirements.txt` file no_deps (bool): If set to True, dependencies will be downloaded/installed without dependencies. Default False. """ self.is_valid = False self.python_dependencies.set_requirements(requirements_path) if no_deps: self.no_deps = True _logger.warning("WARNING! Requiring packages without their transitive dependencies is only supported on AI Inference Server version >=2.6.0") _logger.warning("WARNING! Requiring packages without their transitive dependencies may lead to missing dependencies and runtime errors. Use with caution!") ``` ### `set_pyproject_toml(pyproject_path)` Reads the defined dependencies from the given `pyproject.toml` file and adds it to the requirements list. Only the dependencies defined in the `[project]` section will be added to the component. Parameters: | Name | Type | Description | Default | | ---------------- | ----- | ------------------------------------- | ---------- | | `pyproject_path` | `str` | Path of the given pyproject.toml file | *required* | Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/python_component.py` ```python def set_pyproject_toml(self, pyproject_path: os.PathLike): """ Reads the defined dependencies from the given `pyproject.toml` file and adds it to the requirements list. Only the dependencies defined in the `[project]` section will be added to the component. Args: pyproject_path (str): Path of the given `pyproject.toml` file """ self.is_valid = False self.python_dependencies.set_pyproject_toml(pyproject_path) ``` ### `add_python_packages(path)` Adds Python package(s) to the `PythonPackages.zip` file of the component. The `path` parameter can refer to either a `whl`, a `zip` or a `tar.gz` file. Zip files can be either a source distribution package or a collection of Python packages. Only pure Python source distributions are allowed. The dependency list of the component will be extended with the files added here, so that they will also get installed on the AI Inference Server. The method uses the `tempfile.tempdir` folder, so make sure that the folder is writeable. The wheel files must fulfill the requirements of the targeted device environment (e.g., the Python version must match the supported Python version of the targeted AI Inference Server, and the platform should be one of the supported ones too). Parameters: | Name | Type | Description | Default | | ------ | ----- | ----------------------------- | ---------- | | `path` | `str` | Path of the distribution file | *required* | Examples: `component.add_python_packages('../resources/my_package-0.0.1-py3-none-any.wheel')` adds the wheel file to `PythonPackages.zip` and adds dictionary item `component.dependencies['my_package'] = '0.0.1'` `component.add_python_packages('../resources/inference-wheels.zip')` adds all the wheel files in the zip to `PythonPackages.zip` and `component.dependencies` Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/python_component.py` ```python def add_python_packages(self, path: str) -> None: """ Adds Python package(s) to the `PythonPackages.zip` file of the component. The `path` parameter can refer to either a `whl`, a `zip` or a `tar.gz` file. Zip files can be either a source distribution package or a collection of Python packages. Only pure Python source distributions are allowed. The dependency list of the component will be extended with the files added here, so that they will also get installed on the AI Inference Server. The method uses the `tempfile.tempdir` folder, so make sure that the folder is writeable. The wheel files must fulfill the requirements of the targeted device environment (e.g., the Python version must match the supported Python version of the targeted AI Inference Server, and the platform should be one of the supported ones too). Args: path (str): Path of the distribution file Examples: `component.add_python_packages('../resources/my_package-0.0.1-py3-none-any.wheel')` adds the wheel file to `PythonPackages.zip` and adds dictionary item `component.dependencies['my_package'] = '0.0.1'` `component.add_python_packages('../resources/inference-wheels.zip')` adds all the wheel files in the zip to `PythonPackages.zip` and `component.dependencies` """ self.is_valid = False self.python_dependencies.add_python_packages(path) ``` ### `set_parallel_steps(replicas)` Sets the number of parallel executors. This method configures how many instances of the component can be executed at the same time. The component must be suitable for parallel execution. The inputs arriving to the component will be processed by different instances in parallel, and these instances do not share their state (e.g. variables). Every instance is initialized separately and receives only a fraction of the inputs. AI Inference Server supports at most 8 parallel instances.\`\`\` Parameters: | Name | Type | Description | Default | | ---------- | ----- | ------------------------------------------- | ---------- | | `replicas` | `int` | Number of parallel executors. Default is 1. | *required* | Raises: | Type | Description | | ------------ | ------------------------------------------------ | | `ValueError` | if the given argument is not a positive integer. | Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/python_component.py` ````python def set_parallel_steps(self, replicas): """ Sets the number of parallel executors. This method configures how many instances of the component can be executed at the same time. The component must be suitable for parallel execution. The inputs arriving to the component will be processed by different instances in parallel, and these instances do not share their state (e.g. variables). Every instance is initialized separately and receives only a fraction of the inputs. AI Inference Server supports at most 8 parallel instances.``` Args: replicas (int): Number of parallel executors. Default is 1. Raises: ValueError: if the given argument is not a positive integer. """ self.is_valid = False if (not isinstance(replicas, int)) or replicas < 1: raise ValueError("Replica count must be a positive integer.") if 8 < replicas: _logger.warning("The current maximum of parallel executors is 8.") self._replicas = replicas ```` ### `add_metric(name, desc=None)` Adds a metric that will be automatically used as a pipeline output. Parameters: | Name | Type | Description | Default | | ------ | ----- | ------------------------------------- | ---------- | | `name` | `str` | Name of the metric. | *required* | | `desc` | `str` | Description of the metric. (optional) | `None` | Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/python_component.py` ```python def add_metric(self, name: str, desc: str | None = None): """ Adds a metric that will be automatically used as a pipeline output. Args: name (str): Name of the metric. desc (str): Description of the metric. (optional) """ if "_" not in name: raise AssertionError("The metric name must contain at least one underscore") if self.metrics is None: self.metrics = {} if name in self.metrics: raise AssertionError(f"Metric '{name}' already exists") self.metrics[name] = {} if desc is not None: self.metrics[name]['desc'] = desc ``` ### `delete_metric(name)` Remove a previously added metric. Parameters: | Name | Type | Description | Default | | ------ | ----- | --------------------------------- | ---------- | | `name` | `str` | Name of the metric to be deleted. | *required* | Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/python_component.py` ```python def delete_metric(self, name: str): """ Remove a previously added metric. Args: name (str): Name of the metric to be deleted. """ if name not in self.metrics: raise AssertionError(f"Component '{self.name}' has no metric '{name}'") self.metrics.pop(name) ``` ### `enable_dependency_optimization()` Allows changing repository URLs to optimize the package size Allows the replacement of the `--index-url` argument during `pip download` to download CPU runtime optimized dependencies only. Enabling this optimization, the present `--index-url` will be prepended to the `--extra-index-url` list, and the Pytorch CPU only repository will be set as the `--index-url`. A warning message will be printed if the repository URL modification was necessary. Some dependencies have both CPU and GPU runtime versions, pytorch for example, but a `PythonComponent` can only run on CPU, so packaging the additional GPU runtime dependencies just enlarges the package size. If you want to run your model on GPU, convert it to an `ONNX` model and use it within a `GPURuntimeComponent`. Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/python_component.py` ```python def enable_dependency_optimization(self): """ Allows changing repository URLs to optimize the package size Allows the replacement of the `--index-url` argument during `pip download` to download CPU runtime optimized dependencies only. Enabling this optimization, the present `--index-url` will be prepended to the `--extra-index-url` list, and the Pytorch CPU only repository will be set as the `--index-url`. A warning message will be printed if the repository URL modification was necessary. Some dependencies have both CPU and GPU runtime versions, pytorch for example, but a `PythonComponent` can only run on CPU, so packaging the additional GPU runtime dependencies just enlarges the package size. If you want to run your model on GPU, convert it to an `ONNX` model and use it within a `GPURuntimeComponent`. """ self.python_dependencies.enable_dependency_optimization() ``` ### `disable_dependency_optimization()` Disables any modification to repository URLs Disables the replacement of the `--index-url` argument during `pip download`. This way all `--index-url` or `--extra-index-url` arguments will be preserved if they were present in the requirements.txt file. Some dependencies have both CPU and GPU runtime versions, pytorch for example, but a `PythonComponent` can only run on CPU, so packaging the additional GPU runtime dependencies just enlarges the package size. A warning message will be printed about the package size if this optimization is disabled and the dependency list contains GPU optimized dependencies. Disabling this optimization will not allow the component to run on GPU. If you want to run your model on GPU, convert it to an `ONNX` model and use it within a `GPURuntimeComponent`. Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/python_component.py` ```python def disable_dependency_optimization(self): """ Disables any modification to repository URLs Disables the replacement of the `--index-url` argument during `pip download`. This way all `--index-url` or `--extra-index-url` arguments will be preserved if they were present in the requirements.txt file. Some dependencies have both CPU and GPU runtime versions, pytorch for example, but a `PythonComponent` can only run on CPU, so packaging the additional GPU runtime dependencies just enlarges the package size. A warning message will be printed about the package size if this optimization is disabled and the dependency list contains GPU optimized dependencies. Disabling this optimization will not allow the component to run on GPU. If you want to run your model on GPU, convert it to an `ONNX` model and use it within a `GPURuntimeComponent`. """ self.python_dependencies.disable_dependency_optimization() ``` ### `validate()` Validates that the component is ready to be serialized and packaged as part of a pipeline. Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/python_component.py` ```python def validate(self): """ Validates that the component is ready to be serialized and packaged as part of a pipeline. """ if not self.is_valid: if self.entrypoint is None: raise AssertionError("Entrypoint must be defined") if not any(key.name for key, value in self.resources.items() if key.name == self.entrypoint.name and value == '.'): raise AssertionError("Entrypoint must be added as resource to the root directory before setting up as entrypoint.") if len(self.python_dependencies.dependencies) < 1: _logger.warning(f"WARNING! There are no dependencies defined for component '{self.name}'. Please make sure that all necessary dependencies have been added.") self.python_dependencies.validate() self.is_valid = True _logger.info(f"Component '{self.name}' is valid and ready to use.") ``` ### `save(destination, validate=True)` Saves the component to a folder structure, so it can be used as part of a pipeline configuration package. Validation can be skipped by setting parameter `validate` to False. This is useful when the component is already validated and only intended to be saved. The component folder contains the following: - `requirements.txt` with a list of Python dependencies - Entry point script defined by the `entrypoint` attribute of the component - Extra files as added to the specified folders - `PythonPackages.zip` with the wheel binaries for the environment to be installed Parameters: | Name | Type | Description | Default | | ------------- | ------------- | ----------------------------------------------------------------- | ---------- | | `destination` | `path - like` | Target directory to which the component will be saved. | *required* | | `validate` | `bool` | With value True, triggers component validation. Defaults to True. | `True` | Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/python_component.py` ```python def save(self, destination, validate=True): """ Saves the component to a folder structure, so it can be used as part of a pipeline configuration package. Validation can be skipped by setting parameter `validate` to False. This is useful when the component is already validated and only intended to be saved. The component folder contains the following: - `requirements.txt` with a list of Python dependencies - Entry point script defined by the `entrypoint` attribute of the component - Extra files as added to the specified folders - `PythonPackages.zip` with the wheel binaries for the environment to be installed Args: destination (path-like): Target directory to which the component will be saved. validate (bool): With value True, triggers component validation. Defaults to True. """ if validate: self.validate() folder_path = Path(destination) / self.name folder_path.mkdir(parents=True, exist_ok=True) for file_path in self.resources: dir_path = folder_path / self.resources[file_path] os.makedirs(dir_path, exist_ok=True) shutil.copy(file_path, dir_path / file_path.name) self.python_dependencies.save(folder_path) ``` ## `python_version_validator(version)` Checks if Python version string is valid and describes supported version. Only version 3.11 and 3.12 is supported. A patch version is optional and accepted but logs a warning. Accepted syntaxes are: ```text - {major}.{minor} - {major}.{minor}.{patch} ``` Parameters: | Name | Type | Description | Default | | --------- | ----- | --------------------- | ---------- | | `version` | `str` | Python version string | *required* | Raises: | Type | Description | | ------------ | ---------------------------------------- | | `ValueError` | if the provided version is not supported | Source code in `docs/industrial-ai-suite/sdk/simaticai/deploy/python_component.py` ```python def python_version_validator(version: str): """ Checks if Python version string is valid and describes supported version. Only version 3.11 and 3.12 is supported. A patch version is optional and accepted but logs a warning. Accepted syntaxes are: - {major}.{minor} - {major}.{minor}.{patch} Args: version (str): Python version string Raises: ValueError: if the provided version is not supported """ error_message = "The defined python version is not supported. Currently supported Python versions are 3.11 and 3.12. Python version must be specified only with major and minor version, e.g. '3.11'." warning_message = """Required Python version was specified with patch version. Please note that the patch digit of the required Python version is often not taken into account by the Python ecosystem, so there is no guarantee it has the desired effect.""" match str(version).split('.'): case ('3', '11' | '12'): return # no patch version, so no warning case ('3', '11' | '12', _): _logger.warning(warning_message) case _: raise ValueError(error_message) ``` # deployment Packaging ML models for deployment on the AI Inference Server. The AI SDK offers the functionality to create a pipeline configuration package and wrap trained models, which can be converted to an edge configuration package and then uploaded and run on an AI Inference Server on an Industrial Edge device. From a deployment perspective, the inference pipeline can consist of one or more components. This is independent of the logical structure of the inference pipeline. For example, a typical time series pipeline that consists of multiple Scikit Learn pipeline elements can be packaged into a single pipeline component, which includes both a feature extractor and a classifier. Alternatively, you can deploy the same pipeline split into two components, one for the feature extractor and another for the classifier. To keep things simple and less error-prone, a pipeline should have as few components as possible. In many cases, a single component will be sufficient. However, there might be reasons why you might consider using separate components, such as: - You need a different Python environment for different parts of your processing, e.g., you have components requiring conflicting package versions. - You want to exploit parallelism between components without implementing multithreading. - You want to modularize and build your pipeline from a pool of component variants, which you can combine flexibly. The AI SDK allows you to create pipeline components implemented in Python and compose linear pipelines of one or multiple of such components. The API is designed to anticipate future possible types of components that might be based on a different technology than Python, e.g. ONNX or native TensorFlow Serving. Currently, only Python is supported. For a comprehensive overview on how to package ML models in the context of a machine learning workflow, we recommend you refer to the AI SDK User Manual, especially the chapter concerning packaging models into an inference pipeline. We also recommend you follow the project templates for the AI SDK, which provide packaging notebooks as examples, and where source code and saved trained models are organized into a given folder structure. ## `find_dependencies(name, dependencies)` @Deprecated, reason: uses 'pip show' which only works for installed packages on the current platform. Collects all dependencies of the Python module given with its `name` in the current Python environment. All inherited dependencies will be added to the `dependencies` dictionary with the installed version of the module. The method executes an OS command like `python -m pip show scikit-learn`. Parameters: | Name | Type | Description | Default | | -------------- | ------ | ------------------------------------------------------------------------------------------------------- | ---------- | | `name` | `str` | Name of the Python module to be searched through for its dependencies. | *required* | | `dependencies` | `dict` | Dictionary to collect the dependencies with the module name as key, and the installed version as value. | *required* | Returns: | Name | Type | Description | | ------ | ------ | ------------------------------------------------------------------------- | | `dict` | `dict` | The dependencies dictionary with the collected module names and versions. | Source code in `docs/industrial-ai-suite/sdk/simaticai/deployment.py` ```python def find_dependencies(name: str, dependencies: dict) -> dict: """ @Deprecated, reason: uses 'pip show' which only works for installed packages on the current platform. Collects all dependencies of the Python module given with its `name` in the current Python environment. All inherited dependencies will be added to the `dependencies` dictionary with the installed version of the module. The method executes an OS command like `python -m pip show scikit-learn`. Args: name (str): Name of the Python module to be searched through for its dependencies. dependencies (dict): Dictionary to collect the dependencies with the module name as key, and the installed version as value. Returns: dict: The `dependencies` dictionary with the collected module names and versions. """ cmd_line = [sys.executable, '-m', 'pip', 'show', name] result = subprocess.run(cmd_line, stdout=subprocess.PIPE, text=True) if result.returncode != 0: print(f"Dependency {name} is not found and cannot be added.") return dependencies version = None for line in result.stdout.splitlines(): version_matches = _version_matcher.match(line) if version_matches: version = version_matches.groups()[0].strip() transitive_matches = _transitive_matcher.match(line) if transitive_matches: transitives = transitive_matches.groups()[0].split(", ") for dependency in transitives: if dependency not in dependencies: find_dependencies(dependency, dependencies) if name not in dependencies: spec = pep508.Spec(name, [], [('==', version)] if version else [], None) dependencies[name] = spec print("Found:", spec) return dependencies ``` ## `create_delta_package(origin_edge_package_zip_path, new_edge_package_zip_path)` Creates a Delta Edge Configuration Package from two given Edge Configuration Packages. The created Delta Configuration Package is applicable to import into AI Inference Server, if the Original Edge Configuration Package is already imported there. The Delta Configuration Package only contains the additions and modifications in the New Edge Configuration Package compared to the Original one. That also means that no file deletion is possible in a deployed pipeline via this option. Please make sure that both of the given zip files come from a trusted source! Usage: ```python delta_package_path = deployment.create_delta_package('Edge-Config-edge-1.0.0.zip', 'Edge-Config-edge-1.1.0.zip') ``` This method can be used from the command line, too. ```text python -m simaticai create_delta_package ``` Once the package is calculated, you will have an `Edge-Config-edge-delta-1.1.0.zip` file beside the updated package zip file. This package will contain- the three configuration file for the package;- pipeline_config.yml - runtime_config.yml - datalink_metadata.yml - the newly added files, - and the updated files. The package will not contain any information on the deleted files and they will be copied from the original pipeline. **Caution!** *If you change the version of a component in the pipeline, the delta package will contain all the files of the component because AI Inference Server identifies a component with a different version as a different component!* Parameters: | Name | Type | Description | Default | | ------------------------------ | ----- | ------------------------------------------------------- | ---------- | | `origin_edge_package_zip_path` | `str` | Path to the origin edge configuration package zip file. | *required* | | `new_edge_package_zip_path` | `str` | Path to the new edge configuration package zip file. | *required* | Returns: | Type | Description | | ---------- | ----------------------------------------------------------------- | | `PathLike` | os.PathLike: The path of the created delta edge package zip file. | ```text AssertionError: When: - either of the given edge packages is a delta package or - the names of the given edge packages differ or - the versions of the given edge packages are equal. ``` Source code in `docs/industrial-ai-suite/sdk/simaticai/deployment.py` ````python def create_delta_package(origin_edge_package_zip_path: str, new_edge_package_zip_path: str) -> os.PathLike: """ Creates a Delta Edge Configuration Package from two given Edge Configuration Packages. The created Delta Configuration Package is applicable to import into AI Inference Server, if the Original Edge Configuration Package is already imported there. The Delta Configuration Package only contains the additions and modifications in the New Edge Configuration Package compared to the Original one. That also means that no file deletion is possible in a deployed pipeline via this option. Please make sure that both of the given zip files come from a trusted source! Usage: ~~~python delta_package_path = deployment.create_delta_package('Edge-Config-edge-1.0.0.zip', 'Edge-Config-edge-1.1.0.zip') ~~~ This method can be used from the command line, too. ``` python -m simaticai create_delta_package ``` Once the package is calculated, you will have an `Edge-Config-edge-delta-1.1.0.zip` file beside the updated package zip file.
    This package will contain
    • the three configuration file for the package;
    • pipeline_config.yml
    • runtime_config.yml
    • datalink_metadata.yml
  • the newly added files,
  • and the updated files.
The package will not contain any information on the deleted files and they will be copied from the original pipeline. **Caution!** *If you change the version of a component in the pipeline, the delta package will contain all the files of the component because AI Inference Server identifies a component with a different version as a different component!* Args: origin_edge_package_zip_path (str): Path to the origin edge configuration package zip file. new_edge_package_zip_path (str): Path to the new edge configuration package zip file. Returns: os.PathLike: The path of the created delta edge package zip file. Raises: AssertionError: When: - either of the given edge packages is a delta package or - the names of the given edge packages differ or - the versions of the given edge packages are equal. """ try: # checking whether the user has made a statement about telemetry collection is_telemetry_allowed() except RuntimeError as consent_is_undecided: raise consent_is_undecided workdir = Path(tempfile.mkdtemp(prefix="aisdk_deltapack-")) delta_dir = Path(workdir / "delta") delta_dir.mkdir(parents=True) origin_dir = _extract_edge_package(origin_edge_package_zip_path, Path(workdir / "orig")) new_dir = _extract_edge_package(new_edge_package_zip_path, Path(workdir / "new")) origin_package_info = _get_pipeline_info(origin_dir / PIPELINE_CONFIG) new_package_info = _get_pipeline_info(new_dir / PIPELINE_CONFIG) _validate_delta_package_inputs(origin_package_info, new_package_info) files_in_new_package = new_dir.rglob("*") for f in files_in_new_package: if f.is_dir(): continue orig_file_path = origin_dir / f.relative_to(new_dir) if not orig_file_path.exists(): _copy_file(f, new_dir, delta_dir) else: checksum_original = calc_sha(orig_file_path) checksum_new = calc_sha(f) if checksum_original != checksum_new: _copy_file(f, new_dir, delta_dir) _change_pipeline_config(delta_dir / PIPELINE_CONFIG, origin_package_info["dataFlowPipelineVersion"]) new_edge_package_zip_path = Path(new_edge_package_zip_path) delta_path = _zip_delta_package(delta_dir, new_edge_package_zip_path) shutil.rmtree(workdir, ignore_errors=True) return Path(delta_path) ```` # helpers Helpers. This module contains functionality that does not belong to the main domain of the `simaticai` module. ## `TensorRTOptimization` Class representing TensorRT optimization configuration. This class provides methods to configure the optimization parameters for TensorRT. Attributes: | Name | Type | Description | | -------------------- | ------ | -------------------------------------------------- | | `allowed_parameters` | `list` | List of allowed parameter names. | | `parameters` | `dict` | Dictionary containing the optimization parameters. | Methods: | Name | Description | | --------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `add_extra_parameter` | str, value: str) Adds an extra parameter to the TensorRT optimization. allowed_parameters: "precision_mode", "trt_engine_cache_enable", "trt_engine_cache_path", "max_cached_engines", "minimum_segment_size", "max_workspace_size_bytes" | Intended usage gpu_accelerator = TensorRTOptimization(precision_mode = TensorRTOptimization.PrecisionMode.FP16) .add_extra_parameter("minimum_segment_size", 3) model_config = ModelConfig(model_config, max_batch_size = 1, optimization = gpu_accelerator) Source code in `docs/industrial-ai-suite/sdk/simaticai/helpers/model_config.py` ```python class TensorRTOptimization: """ Class representing TensorRT optimization configuration. This class provides methods to configure the optimization parameters for TensorRT. Attributes: allowed_parameters (list): List of allowed parameter names. parameters (dict): Dictionary containing the optimization parameters. Methods: add_extra_parameter(self, key: str, value: str) Adds an extra parameter to the TensorRT optimization. allowed_parameters: "precision_mode", "trt_engine_cache_enable", "trt_engine_cache_path", "max_cached_engines", "minimum_segment_size", "max_workspace_size_bytes" Intended usage: gpu_accelerator = TensorRTOptimization(precision_mode = TensorRTOptimization.PrecisionMode.FP16) .add_extra_parameter("minimum_segment_size", 3) model_config = ModelConfig(model_config, max_batch_size = 1, optimization = gpu_accelerator) """ class PrecisionMode(Enum): """ Enum class for different precision modes in TensorRT optimization. """ FP32 = "FP32" FP16 = "FP16" allowed_parameters = ["precision_mode", "max_cached_engines", "minimum_segment_size", "max_workspace_size_bytes", "trt_engine_cache_enable", "trt_engine_cache_path" ] def __init__(self, precision_mode: PrecisionMode = PrecisionMode.FP32): """ Initializes a new instance of the TensorRTOptimization class. Args: precision_mode (PrecisionMode): The precision mode for the TensorRT optimization. """ self.parameters = { "precision_mode": precision_mode.value, "trt_engine_cache_enable": "true", "trt_engine_cache_path": "/tmp/triton" } def add_extra_parameter(self, key: str, value: str): """ Add extra parameter to the TensorRT optimization. Args: key (str): The key of the parameter. value (str): The value of the parameter. """ assert key in self.allowed_parameters, f"Parameter '{key}' is not allowed" if key in self.parameters: _logger.warn(f"Parameter '{key}' already exists with value {self.parameters[key]} and will be overwritten with value {value}") self.parameters[key] = value return self def __str__(self): """ Returns a string representation of the TensorRTOptimization object. """ return TPL_TENSORRT_ACCELERATOR.format(extra_parameters=self._parameters_to_string()) def _parameters_to_string(self): """ Converts the parameters dictionary to a string representation. """ return "\n\t\t".join([f"parameters {{ key: \"{key}\" value: \"{value}\" }}" for key, value in self.parameters.items()]) ``` ### `PrecisionMode` Bases: `Enum` Enum class for different precision modes in TensorRT optimization. Source code in `docs/industrial-ai-suite/sdk/simaticai/helpers/model_config.py` ```python class PrecisionMode(Enum): """ Enum class for different precision modes in TensorRT optimization. """ FP32 = "FP32" FP16 = "FP16" ``` ### `__init__(precision_mode=PrecisionMode.FP32)` Initializes a new instance of the TensorRTOptimization class. Parameters: | Name | Type | Description | Default | | ---------------- | --------------- | ------------------------------------------------- | ------- | | `precision_mode` | `PrecisionMode` | The precision mode for the TensorRT optimization. | `FP32` | Source code in `docs/industrial-ai-suite/sdk/simaticai/helpers/model_config.py` ```python def __init__(self, precision_mode: PrecisionMode = PrecisionMode.FP32): """ Initializes a new instance of the TensorRTOptimization class. Args: precision_mode (PrecisionMode): The precision mode for the TensorRT optimization. """ self.parameters = { "precision_mode": precision_mode.value, "trt_engine_cache_enable": "true", "trt_engine_cache_path": "/tmp/triton" } ``` ### `add_extra_parameter(key, value)` Add extra parameter to the TensorRT optimization. Parameters: | Name | Type | Description | Default | | ------- | ----- | --------------------------- | ---------- | | `key` | `str` | The key of the parameter. | *required* | | `value` | `str` | The value of the parameter. | *required* | Source code in `docs/industrial-ai-suite/sdk/simaticai/helpers/model_config.py` ```python def add_extra_parameter(self, key: str, value: str): """ Add extra parameter to the TensorRT optimization. Args: key (str): The key of the parameter. value (str): The value of the parameter. """ assert key in self.allowed_parameters, f"Parameter '{key}' is not allowed" if key in self.parameters: _logger.warn(f"Parameter '{key}' already exists with value {self.parameters[key]} and will be overwritten with value {value}") self.parameters[key] = value return self ``` ### `__str__()` Returns a string representation of the TensorRTOptimization object. Source code in `docs/industrial-ai-suite/sdk/simaticai/helpers/model_config.py` ```python def __str__(self): """ Returns a string representation of the TensorRTOptimization object. """ return TPL_TENSORRT_ACCELERATOR.format(extra_parameters=self._parameters_to_string()) ``` ## `parse_lines(lines)` Processes dependency requirement lines. Parameters: | Name | Type | Description | Default | | ------- | ----------- | -------------------------------------------- | ---------- | | `lines` | `list[str]` | list of dependencies and (extra) index urls. | *required* | Returns: | Type | Description | | ------------------------------ | --------------------------------------------------------------------------------------- | | `(dict[str, Spec], list[str])` | dictionary that contains the dependency specifcations and a list of (extra) index urls. | Raises: | Type | Description | | ---------------- | ------------------------------------------------------- | | `AssertionError` | if the lines contain invalid dependency specifications. | Source code in `docs/industrial-ai-suite/sdk/simaticai/helpers/pep508.py` ```python def parse_lines(lines: list[str]) -> "(dict[str, Spec], list[str], str)": """ Processes dependency requirement lines. Args: lines (list[str]): list of dependencies and (extra) index urls. Returns: (dict[str, Spec], list[str]): dictionary that contains the dependency specifcations and a list of (extra) index urls. Raises: AssertionError: if the lines contain invalid dependency specifications. """ dependencies = {} invalid_lines = [] extra_index = [] index_url = None lines = map(lambda row: row.strip(), lines) lines = filter(lambda row: row != "" and not row.startswith("#"), lines) for line in lines: if line.startswith('--index-url'): index_url = line continue if line.startswith('--extra-index-url'): extra_index.append(line) continue try: spec: Spec = parse_line(line) dependencies[spec.name] = spec except Exception as e: invalid_lines.append(f"{line}\n{e}") if len(invalid_lines) > 0: raise AssertionError('\n'.join(invalid_lines)) return (dependencies, extra_index, index_url) ``` ## `parse_requirements(requirements_path)` Processes a requirements.txt file line-by-line. Parameters: | Name | Type | Description | Default | | ------------------- | ---------------------- | ------------------------------------------------------------- | ---------- | | `requirements_path` | `Union[str, PathLike]` | Union\[str, os.PathLike\]: path to the requirements.txt file. | *required* | Returns: | Type | Description | | ------------------------------ | --------------------------------------------------------------------------------------- | | `(dict[str, Spec], list[str])` | dictionary that contains the dependency specifcations and a list of (extra) index urls. | Raises: | Type | Description | | ---------------- | ------------------------------------------------------- | | `AssertionError` | if the lines contain invalid dependency specifications. | Source code in `docs/industrial-ai-suite/sdk/simaticai/helpers/pep508.py` ```python def parse_requirements(requirements_path: Union[str, os.PathLike]) -> "(dict[str, Spec], list[str], str)": """ Processes a requirements.txt file line-by-line. Args: requirements_path: Union[str, os.PathLike]: path to the requirements.txt file. Returns: (dict[str, Spec], list[str]): dictionary that contains the dependency specifcations and a list of (extra) index urls. Raises: AssertionError: if the lines contain invalid dependency specifications. """ with open(Path(requirements_path), "r") as f: lines = f.readlines() try: return parse_lines(lines) except AssertionError as err: raise AssertionError(f"Requirements file '{requirements_path}' contains invalid dependency specifications:\n{str(err)}") ``` ## `parse_pyproject_toml(pyproject_path)` Processes a pyproject.toml file line-by-line. Parameters: | Name | Type | Description | Default | | ---------------- | ---------------------- | ----------------------------------------------------------- | ---------- | | `pyproject_path` | `Union[str, PathLike]` | Union\[str, os.PathLike\]: path to the pyproject.toml file. | *required* | Returns: | Type | Description | | ------------------------------ | --------------------------------------------------------------------------------------- | | `(dict[str, Spec], list[str])` | dictionary that contains the dependency specifcations and a list of (extra) index urls. | Raises: | Type | Description | | ---------------- | ------------------------------------------------------- | | `AssertionError` | if the lines contain invalid dependency specifications. | Source code in `docs/industrial-ai-suite/sdk/simaticai/helpers/pep508.py` ```python def parse_pyproject_toml(pyproject_path: Union[str, os.PathLike]) -> "(dict[str, Spec], list[str], str)": """ Processes a pyproject.toml file line-by-line. Args: pyproject_path: Union[str, os.PathLike]: path to the pyproject.toml file. Returns: (dict[str, Spec], list[str]): dictionary that contains the dependency specifcations and a list of (extra) index urls. Raises: AssertionError: if the lines contain invalid dependency specifications. """ with open(Path(pyproject_path), "rb") as f: pyproject = toml.load(f) if 'project' in pyproject and 'dependencies' in pyproject['project']: lines = pyproject['project']['dependencies'] else: raise AssertionError(f"The file '{pyproject_path}' must contain a [project] section with a [dependencies] field.") try: return parse_lines(lines) except AssertionError as err: raise AssertionError(f"The file '{pyproject_path}' contains invalid dependency specifications:\n{str(err)}") ``` Classes to generate a report for a dataflow pipeline and local pipeline runner. ## `ReportWriter` Base class for report writers. Source code in `docs/industrial-ai-suite/sdk/simaticai/helpers/reporter.py` ```python class ReportWriter: """ Base class for report writers. """ def __init__(self): self.report_path = None self.warnings_text = "" self.errors_text = "" def set_path(self, report_path: Path): self.report_path = report_path def add_warning(self, name, filename, line_number, warning_msg): self.warnings_text += WARNING_LINE.format(name=name, filename=filename, line_number=line_number, warning_msg=warning_msg) def add_error(self, name, filename, line_number, error_msg): self.errors_text += ERROR_LINE.format(name=name, filename=filename, line_number=line_number, error_msg=error_msg) def write_report(self): raise NotImplementedError("Subclasses should implement this method") def _write_warnings(self, file): file.write(WARNINGS_HEADLINE) file.write(self.warnings_text) def _write_errors(self, file): file.write(ERRORS_HEADLINE) file.write(self.errors_text) ``` ## `ReportWriterHandler` Bases: `Handler` A handler that can be given to a logger, so the report writer can capture logged warning and error messages Source code in `docs/industrial-ai-suite/sdk/simaticai/helpers/reporter.py` ```python class ReportWriterHandler(logging.Handler): """ A handler that can be given to a logger, so the report writer can capture logged warning and error messages """ def __init__(self, report_writer: ReportWriter): super().__init__() self.report_writer = report_writer def emit(self, record): if record.levelno == logging.WARNING: self.report_writer.add_warning(record.name, record.filename, record.lineno, record.getMessage()) elif record.levelno == logging.ERROR: self.report_writer.add_error(record.name, record.filename, record.lineno, record.getMessage()) ``` ## `ZipTreeElement` A class to represent a file or folder in a zip file. During the recursive traversal of the zip file, the full name and file size are stored in this class. Source code in `docs/industrial-ai-suite/sdk/simaticai/helpers/reporter.py` ```python class ZipTreeElement: """ A class to represent a file or folder in a zip file. During the recursive traversal of the zip file, the full name and file size are stored in this class. """ def __init__(self, full_name, file_size): self.full_name = full_name self.file_size = file_size ``` ## `PipelineReportWriter` Bases: `ReportWriter` A class to generate a report for a dataflow pipeline, including pipeline structure, component dependencies, and package vulnerabilities. Methods: | Name | Description | | ------------------------- | --------------------------------------------------------------------------------------------------------------------------- | | `set_path` | Path): Sets the path where the report will be saved. | | `set_pipeline_config` | dict): Sets the pipeline configuration and updates the pipeline info and structure. | | `add_full_dependency_set` | str, dependency_set: set[tuple]): Adds a full set of dependencies for a component and updates the vulnerability dictionary. | | `add_direct_dependencies` | str, direct_dependencies: dict): Adds direct dependencies for a component. | | `add_warning` | Adds a warning to the report. | | `write_report` | Writes the report to the specified path. | Source code in `docs/industrial-ai-suite/sdk/simaticai/helpers/reporter.py` ````python class PipelineReportWriter(ReportWriter): """ A class to generate a report for a dataflow pipeline, including pipeline structure, component dependencies, and package vulnerabilities. Methods: set_path(report_path: Path): Sets the path where the report will be saved. set_pipeline_config(pipeline_config: dict): Sets the pipeline configuration and updates the pipeline info and structure. add_full_dependency_set(component_name: str, dependency_set: set[tuple]): Adds a full set of dependencies for a component and updates the vulnerability dictionary. add_direct_dependencies(component_name: str, direct_dependencies: dict): Adds direct dependencies for a component. add_warning(name, filename, line_number, warning_msg): Adds a warning to the report. write_report(): Writes the report to the specified path. """ def __init__(self): super().__init__() self.pipeline_config = {} # pipeline config json self.pipeline_name = "Unnamed pipeline" # dependency_names and package_names are transformed dependency names (lowercase, underscore instead of dash) # set from outside self.component_direct_dependency_namelist = {} # component -> list of dependency_names (with NO version) self.component_all_dependencies = {} # component -> set of tuples of (dependency_name, dependency_version) # collected before writing self.component_direct_dependencies = {} # component -> set of dependency_names self.component_transitive_dependencies = {} # component -> set of tuples of (dependency_name, dependency_version) self.vulnerability_dict = {} # (package_name, package_version) -> vulnerabilities (None | list of dictionaries) # report text sections to fill self.pipeline_structure_text = "" self.pipeline_info_text = "" self.warnings_text = "" def set_pipeline_config(self, pipeline_config: dict): self.pipeline_config = pipeline_config self._set_pipeline_info() self._set_pipeline_structure() @staticmethod def _sort_pipeline_dag(pipeline_dag: list) -> list: """ Sorts a pipeline DAG in order to show dataflow from Pipeline Inputs to Pipeline Outputs. Databus component is a privileged source, and it is always the first component in the report. Args: pipeline_dag (list): The pipeline DAG is a list of dictionaries with "source" and "target" keys. Returns: A sorted list of dictionaries representing the pipeline DAG. """ pipeline_dag.sort(key=lambda x: (x["source"], x["target"])) sorted_dag = [edge for edge in pipeline_dag if "Databus" in edge['source']] if sorted_dag == []: return pipeline_dag pipeline_dag = [edge for edge in pipeline_dag if "Databus" not in edge['source']] # Extracts name of the target or source component from the edge name_of_component = lambda edge, target_or_source: edge[target_or_source].rsplit(".", 1)[0] while len(pipeline_dag) > 0: sorted_targets = [name_of_component(edge, "target") for edge in sorted_dag] sorted_dag.extend([ edge for edge in pipeline_dag if name_of_component(edge, "source") in sorted_targets ]) pipeline_dag = [edge for edge in pipeline_dag if edge not in sorted_dag] return sorted_dag def _set_pipeline_structure(self): self.pipeline_name = self.pipeline_config.get("dataFlowPipelineInfo", {}).get("projectName", "n/a") self.pipeline_structure_text = "```plantuml\n" components = self.pipeline_config.get("dataFlowPipeline", {}).get("components", []) variables = {} # name: (type, is_metric) for component in components: input_variables = {_input["name"]: (_input["type"], False) for _input in component.get("inputType")} output_variables = {_output["name"]: (_output["type"], _output.get("metric", False)) for _output in component.get("outputType")} variables.update({**input_variables, **output_variables}) pipeline_dag = self.pipeline_config.get("dataFlowPipeline", {}).get("pipelineDag", []) sorted_pipeline_dag = PipelineReportWriter._sort_pipeline_dag(pipeline_dag) for transition in sorted_pipeline_dag: source_component_name, source_variable_name = transition["source"].rsplit(".", 1) target_component_name, target_variable_name = transition["target"].rsplit(".", 1) variable_name_to_show = source_variable_name if source_variable_name == target_variable_name else f"{source_variable_name} -> {target_variable_name}" source_component_name = source_component_name.replace("Databus", "AIIS") target_component_name = target_component_name.replace("Databus", "AIIS") variable_type, is_metric = variables[source_variable_name] arrow = "-->" if is_metric else "->" # metric variables are drawn with a dashed line self.pipeline_structure_text += PL_STRUCTURE.format(source_component=source_component_name, arrow=arrow, target_component=target_component_name, variable_name=variable_name_to_show, variable_type=variable_type) self.pipeline_structure_text += "```\n\n" def _set_pipeline_info(self): dataflow_pipeline_info = self.pipeline_config.get("dataFlowPipelineInfo", {}) author = dataflow_pipeline_info.get("author", "n/a") created_on = dataflow_pipeline_info.get("createdOn", "n/a") pipeline_version = dataflow_pipeline_info.get("dataFlowPipelineVersion", "n/a") description = dataflow_pipeline_info.get("description", "n/a") package_id = dataflow_pipeline_info.get("packageId", "n/a") project_name = dataflow_pipeline_info.get("projectName", "n/a") self.pipeline_info_text = PL_INFO.format(author=author, created_on=created_on, pipeline_version=pipeline_version, description=description, package_id=package_id, project_name=project_name) # Transform every dependency and package name for consistency; i.e., # opencv-python-headless -> opencv_python_headless; Django -> django @staticmethod def transform_package_name(name: str): new_name = name.replace("-", "_") return new_name.lower() # A full dependency set is a set of (package_name, package_version) tuples # and contains all the dependencies installed for a component def add_full_dependency_set(self, component_name: str, dependency_set: set[tuple]): dependency_list = sorted(list(dependency_set), key=lambda x: x[0]) self._expand_component_all_dependencies(component_name, dependency_list) self._update_vulnerability_dict(dependency_list) def _expand_component_all_dependencies(self, component_name: str, dependency_list: list[tuple]): if component_name not in self.component_all_dependencies: self.component_all_dependencies[component_name] = set() for package_name, package_version in dependency_list: transformed_package_name = PipelineReportWriter.transform_package_name(package_name) self.component_all_dependencies[component_name].add((transformed_package_name, package_version)) def _update_vulnerability_dict(self, dependency_list: list[tuple]): vulnerability_dict = {} for package_name, package_version in dependency_list: transformed_package_name = PipelineReportWriter.transform_package_name(package_name) vulnerability_dict[(transformed_package_name, package_version)] = None url = f"https://pypi.org/pypi/{package_name}/{package_version}/json" try: response = requests.get(url, timeout=5) if response.status_code == 200: data = response.json() if 'vulnerabilities' in data: vulnerability_dict[(transformed_package_name, package_version)] = data['vulnerabilities'] except requests.exceptions.Timeout: pass self.vulnerability_dict.update(vulnerability_dict) def add_direct_dependencies(self, component_name: str, direct_dependencies: dict): self.component_direct_dependency_namelist[component_name] = [PipelineReportWriter.transform_package_name(name) for name in list(direct_dependencies.keys())] def write_report(self): if self.report_path is None: return self._set_component_dependencies() with open(self.report_path, "w", encoding="utf-8") as file: self._write_headline(file) self._write_pipeline_info(file) self._write_pipeline_structure(file) self._write_dependencies(file) self._write_package_vulnerabilities(file) self._write_errors(file) self._write_warnings(file) def _set_component_dependencies(self): for component in self.component_all_dependencies.keys(): # self.component_direct_dependencies should contain everything from self.component_all_dependencies # if it is direct, i.e., the name is in self.component_direct_dependency_namelist # self.component_transitive_dependencies should contain everything else self.component_transitive_dependencies[component] = set() self.component_direct_dependencies[component] = set() all_dependencies = self.component_all_dependencies[component] for dependency_name, dependency_version in all_dependencies: if dependency_name in self.component_direct_dependency_namelist.get(component, []): self.component_direct_dependencies[component].add((dependency_name, dependency_version)) else: self.component_transitive_dependencies[component].add((dependency_name, dependency_version)) def _write_headline(self, file): file.write(PL_REPORT_HEADLINE.format(pipeline_name=self.pipeline_name)) def _write_pipeline_info(self, file): file.write(PL_INFO_HEADLINE) file.write(self.pipeline_info_text) def _write_pipeline_structure(self, file): file.write(PL_STRUCTURE_HEADLINE) file.write(self.pipeline_structure_text) def _write_dependencies(self, file): for component_name in self.component_all_dependencies.keys(): direct_dependencies = self.component_direct_dependencies.get(component_name, set()) transitive_dependencies = self.component_transitive_dependencies.get(component_name, set()) file.write(PL_COMPONMENT_DEPENDENCIES_HEADLINE.format(component_name=component_name)) file.write(PL_COMPONENT_DIRECT_DEPENDENCIES_HEADLINE) sorted_direct_dependencies = sorted(list(direct_dependencies), key=lambda x: x[0]) for dependency_name, dependency_version in sorted_direct_dependencies: file.write(PL_COMPONENT_DIRECT_DEPENDENCY.format(dependency_name=dependency_name, dependency_version=dependency_version)) file.write("\n") file.write(PL_COMPONENT_TRANSITIVE_DEPENDENCIES_HEADLINE) sorted_transitive_dependencies = sorted(list(transitive_dependencies), key=lambda x: x[0]) for dependency_name, dependency_version in sorted_transitive_dependencies: file.write(PL_COMPONENT_TRANSITIVE_DEPENDENCY.format(dependency_name=dependency_name, dependency_version=dependency_version)) file.write("\n") def _get_components_who_have_given_package(self, package_name, package_version): components = [] for component in self.component_all_dependencies: dependencies = self.component_all_dependencies[component] if (package_name, package_version) in dependencies: components.append(component) return components def _write_package_vulnerabilities(self, file): file.write(PL_PACKAGE_VULNERABILITIES_HEADLINE) sorted_vulnerability_dict_items = sorted(self.vulnerability_dict.items(), key=lambda x: x[0][0]) for (package_name, package_version), vulnerabilities in sorted_vulnerability_dict_items: components = ', '.join(self._get_components_who_have_given_package(package_name, package_version)) if vulnerabilities is None: file.write(PL_PACKAGE_VULNERABILITY_CANNOT_BE_CHECKED.format(package_name=package_name, package_version=package_version, components=components)) elif vulnerabilities == []: file.write(PL_PACKAGE_VULNERABILITY_NOT_KNOWN.format(package_name=package_name, package_version=package_version, components=components)) else: for vulnerability in vulnerabilities: vulnerability_aliases = vulnerability.get('aliases', 'Vulnerability found with no alias. Check [PyPI repository](https://pypi.org/) for more details.') vulnerability_link = vulnerability.get('link', 'No link found') if vulnerability_link != 'No link found': vulnerability_link = f"[{vulnerability_link}]({vulnerability_link})" vulnerability_details = vulnerability.get('details', 'No details found') vulnerability_fixed_in = vulnerability.get('fixed_in', '') file.write(PL_PACKAGE_VULNERABILITY.format(package_name=package_name, package_version=package_version, vulnerability_aliases=vulnerability_aliases, vulnerability_link=vulnerability_link, vulnerability_details=vulnerability_details, vulnerability_fixed_in=vulnerability_fixed_in, components=components)) file.write("\n") ```` ## `PipelineRunnerReportWriter` Bases: `ReportWriter` PipelineRunnerReportWriter is responsible for generating a detailed report of a local pipeline execution. It builds folder structures from zip files, manages component payload counts, and adds installed packages information. Methods: | Name | Description | | --------------------------- | -------------------------------------------------------------------------------------------- | | `set_path` | Path): Sets the path where the report will be saved. | | `set_package_zip_path` | Path): Sets the path to the package zip file and updates the folder tree. | | `set_input_payload_length` | str, length: int): Sets the input payload length for a component. | | `set_output_payload_length` | str, length: int): Sets the output payload length for a component. | | `add_installed_packages` | str, pip_report_file: Path): Adds installed packages for a component from a pip report file. | | `add_warning` | Adds a warning to the report. | | `write_report` | Writes the report to the specified path. | Source code in `docs/industrial-ai-suite/sdk/simaticai/helpers/reporter.py` ```python class PipelineRunnerReportWriter(ReportWriter): """ PipelineRunnerReportWriter is responsible for generating a detailed report of a local pipeline execution. It builds folder structures from zip files, manages component payload counts, and adds installed packages information. Methods: set_path(report_path: Path): Sets the path where the report will be saved. set_package_zip_path(zip_path: Path): Sets the path to the package zip file and updates the folder tree. set_input_payload_length(component_name: str, length: int): Sets the input payload length for a component. set_output_payload_length(component_name: str, length: int): Sets the output payload length for a component. add_installed_packages(component_name: str, pip_report_file: Path): Adds installed packages for a component from a pip report file. add_warning(name, filename, line_number, warning_msg): Adds a warning to the report. write_report(): Writes the report to the specified path. """ def __init__(self): super().__init__() self.package_zip_path = None self.zip_file_name = "" self.component_installed_packages = {} # component_name -> list[tuple(package_name, package_version, whl_name)] self.component_payload_length = {} # component_name -> [input_payload_length, output_payload_length] self.python_packages_zip_content = set() # report text sections to fill self.folder_tree_text = "" self.warnings_text = "" def set_package_zip_path(self, zip_path: Path): self.package_zip_path = zip_path with zipfile.ZipFile(zip_path, 'r') as zipf: self.zip_file_name = zipf.filename zip_tree = {} for item_name in zipf.namelist(): zip_tree[item_name] = ZipTreeElement(full_name=item_name, file_size=zipf.getinfo(item_name).file_size) self._print_structure_recursively(zip_tree, zipf) @staticmethod def _get_folder_and_file_list(item_names: list) -> tuple[list, list]: """ Given a list of item names, each item name is a file that either starts with a folder name, or not. This function separates the folder names and the standalone file names. E.g., ["a/b/something.txt", "c/another.txt", "else.txt"] -> ["a/", "c/"], ["else.txt"] """ folder_names = set() file_names = [] for item in item_names: item_parts = item.split('/') if len(item_parts) > 1: if item_parts[0] != '': folder_names.add(item_parts[0] + '/') else: if item != '': file_names.append(item) return sorted(list(folder_names)), sorted(file_names) @staticmethod def format_size(size): """Format file size in human-readable form.""" for unit in ['B', 'KB', 'MB']: if size < 1000: return f"{size} {unit}" size //= 1000 return f"{size} GB" def _print_structure_recursively(self, zip_tree, zipf, prefix=""): folder_names, file_names = PipelineRunnerReportWriter._get_folder_and_file_list(zip_tree.keys()) is_file_names_empty = file_names == [] self._print_folder_structure(zip_tree, zipf, prefix, folder_names, is_file_names_empty) self._print_file_structure(zip_tree, zipf, prefix, file_names) def _print_folder_structure(self, zip_tree, zipf, prefix, folder_names, is_file_names_empty): for i, folder in enumerate(folder_names): is_last = (i == len(folder_names) - 1) and is_file_names_empty connector = LPLR_FOLDER_STRUCTURE_LAST_CONNECTOR_SYMBOL if is_last else LPLR_FOLDER_STRUCTURE_MID_CONNECTOR_SYMBOL self.folder_tree_text += LPLR_FOLDER_STRUCTURE_FOLDER_LINE.format(prefix=prefix, connector=connector, folder=folder) # create a new tree where items start with the same folder name; but cut out the folder name new_zip_tree_from_folder = {} for k, v in zip_tree.items(): if k.startswith(folder): new_file_name = k.split('/', 1)[1] new_zip_tree_from_folder[new_file_name] = v prefix_post = LPLR_FOLDER_STRUCTURE_LAST_PREFIX_SYMBOL if is_last else LPLR_FOLDER_STRUCTURE_MID_PREFIX_SYMBOL new_prefix_from_folder = prefix + prefix_post self._print_structure_recursively(new_zip_tree_from_folder, zipf, new_prefix_from_folder) def _print_file_structure(self, zip_tree, zipf, prefix, file_names): for i, file_name in enumerate(file_names): is_last = (i == len(file_names) - 1) size_str = PipelineRunnerReportWriter.format_size(zip_tree[file_name].file_size) connector = LPLR_FOLDER_STRUCTURE_LAST_CONNECTOR_SYMBOL if is_last else LPLR_FOLDER_STRUCTURE_MID_CONNECTOR_SYMBOL self.folder_tree_text += LPLR_FOLDER_STRUCTURE_FILE_LINE.format(prefix=prefix, connector=connector, file=file_name, size=size_str) # zip files are handled similarly to folders: # create a new tree where items start with the same folder name; but cut out the folder name if not file_name.endswith('.zip'): continue full_name = zip_tree[file_name].full_name with zipf.open(full_name) as nested_zip_file: nested_zip_data = io.BytesIO(nested_zip_file.read()) with zipfile.ZipFile(nested_zip_data, 'r') as nested_zipf: if file_name == "PythonPackages.zip": self.python_packages_zip_content.update(sorted(list(nested_zipf.namelist()))) # create a new tree where items start with the same folder name; but cut out the folder name new_zip_tree_from_zip = {} for nested_item_name in nested_zipf.namelist(): new_zip_tree_from_zip[nested_item_name] = ZipTreeElement(nested_item_name, nested_zipf.getinfo(nested_item_name).file_size) prefix_post = LPLR_FOLDER_STRUCTURE_LAST_PREFIX_SYMBOL if is_last else LPLR_FOLDER_STRUCTURE_MID_PREFIX_SYMBOL new_prefix_from_zip = prefix + prefix_post self._print_structure_recursively(new_zip_tree_from_zip, nested_zipf, new_prefix_from_zip) def set_input_payload_length(self, component_name: str, length: int): if component_name in self.component_payload_length: self.component_payload_length[component_name][0] = length else: self.component_payload_length[component_name] = [length, 0] def set_output_payload_length(self, component_name: str, length: int): if component_name in self.component_payload_length: self.component_payload_length[component_name][1] = length else: self.component_payload_length[component_name] = [0, length] def add_installed_packages(self, component_name: str, pip_report_file: Path): if not pip_report_file.is_file(): return pip_report = {} with open(pip_report_file, 'r', encoding="utf-8") as file: pip_report = json.load(file) if component_name not in self.component_installed_packages: self.component_installed_packages[component_name] = [] installed_packages = pip_report.get("install", []) for package in installed_packages: package_url = package.get("download_info", {}).get("url", "") wheel_name = package_url.split("/")[-1] if package_url.endswith(".whl") else "n/a" metadata = package.get("metadata", {}) package_name = metadata.get("name", "n/a") package_version = metadata.get("version", "n/a") self.component_installed_packages[component_name].append((package_name, package_version, wheel_name)) # check if one or more reports already exists; set the report path so a new report will have a new index def _set_path_from_zip_path(self): if self.package_zip_path is None: return workdir = self.package_zip_path.parent base_name = self.package_zip_path.stem report_files = list(workdir.glob(f"{base_name}_execution_report_*.md")) max_index = 0 for report_file in report_files: try: index = int(report_file.stem.split('_')[-1]) if index > max_index: max_index = index except ValueError: continue self.set_path(workdir / f"{base_name}_execution_report_{max_index + 1}.md") def write_report(self): # if the report path is not set, set it from the zip path self._set_path_from_zip_path() if self.report_path is None: return with open(self.report_path, "w", encoding="utf-8") as file: self._write_headline(file) self._write_folder_structure(file) self._write_python_packages_zip_content(file) self._write_component_installed_packages(file) self._write_payload_lengths(file) self._write_errors(file) self._write_warnings(file) def _write_headline(self, file): file.write(LPLR_REPORT_HEADLINE) def _write_folder_structure(self, file): file.write(LPLR_FOLDER_STRUCTURE_HEADLINE.format(file_name=self.zip_file_name)) file.write(LPLR_FOLDER_STRUCTURE.format(file_name=self.zip_file_name, folder_structure=self.folder_tree_text)) def _write_python_packages_zip_content(self, file): file.write(LPLR_PYTHON_PACKAGES_ZIP_CONTENT_HEADLINE) sorted_zip_content = sorted(list(self.python_packages_zip_content)) for package in sorted_zip_content: file.write(LPLR_PYTHON_PACKAGES_ZIP_CONTENT.format(python_package=package)) file.write("\n") def _write_component_installed_packages(self, file): file.write(LPLR_COMPONENT_INSTALLED_PACKAGES_HEADLINE) for component in self.component_installed_packages: file.write(LPLR_COMPONENT_INSTALLED_PACKAGES.format(component=component)) sorted_installed_packages = sorted(self.component_installed_packages[component], key=lambda x: x[0]) for package_name, package_version, wheel_name in sorted_installed_packages: file.write(LPLR_COMPONENT_INSTALLED_PACKAGES_ROW.format(package_name=package_name, package_version=package_version, wheel_name=wheel_name)) file.write("\n") def _write_payload_lengths(self, file): file.write(LPLR_PAYLOAD_LENGTHS_HEADLINE) for component in self.component_payload_length: input_payload_length, output_payload_length = self.component_payload_length[component] file.write(LPLR_PAYLOAD_LENGTHS.format(component=component, input_payload_length=input_payload_length, output_payload_length=output_payload_length)) ``` ### `format_size(size)` Format file size in human-readable form. Source code in `docs/industrial-ai-suite/sdk/simaticai/helpers/reporter.py` ```python @staticmethod def format_size(size): """Format file size in human-readable form.""" for unit in ['B', 'KB', 'MB']: if size < 1000: return f"{size} {unit}" size //= 1000 return f"{size} GB" ``` Module for dealing with temporary files. This module helps with extracting a zip file into a temporary folder. ## `OpenZipInTemp` Unzip a zip archive into a temporary directory. Example usage: ```text with OpenZipInTemp("path_to_zip_file.zip") as temp_dir: # do something with temp_dir pass ``` Parameters: | Name | Type | Description | Default | | ---------- | ------------- | -------------------- | ---------- | | `zip_path` | `path - like` | path to the archive. | *required* | Source code in `docs/industrial-ai-suite/sdk/simaticai/helpers/tempfiles.py` ```python class OpenZipInTemp: """ Unzip a zip archive into a temporary directory. Example usage: with OpenZipInTemp("path_to_zip_file.zip") as temp_dir: # do something with temp_dir pass Args: zip_path (path-like): path to the archive. """ def __init__(self, zip_path: Union[str, os.PathLike], clean_up: bool = True): if not zipfile.is_zipfile(zip_path): raise ValueError(f"File does not exist or not a zip file: {zip_path}") self.zip_path = zip_path self.tmp_path = None self.clean_up = clean_up def __enter__(self) -> Path: self.tmp_path = Path(tempfile.mkdtemp(prefix="unzip-")) with zipfile.ZipFile(self.zip_path, "r") as zip_file: zip_file.extractall(path=self.tmp_path) return self.tmp_path def __exit__(self, e_type, e_val, e_trace): if self.clean_up: shutil.rmtree(self.tmp_path, ignore_errors=True) ``` Helper module for YAML files. Reads YAML files into a dictionary with a custom loader. ## `read_yaml(path)` Read a YAML file into a dictionary. Loads the YAML file specified by `path`. The YAML loader is configured to read datetime objects as strings, for simplifying validation with a JSON schema. Parameters: | Name | Type | Description | Default | | ------ | ------------- | ---------------------- | ---------- | | `path` | `path - like` | Path of the YAML file. | *required* | Returns: | Name | Type | Description | | ------ | ------ | ------------------------------------------- | | `dict` | `dict` | A dictionary, populated from the YAML file. | Source code in `docs/industrial-ai-suite/sdk/simaticai/helpers/yaml_helper.py` ```python def read_yaml(path: os.PathLike | str) -> dict: """ Read a YAML file into a dictionary. Loads the YAML file specified by `path`. The YAML loader is configured to read datetime objects as strings, for simplifying validation with a JSON schema. Args: path (path-like): Path of the YAML file. Returns: dict: A dictionary, populated from the YAML file. """ _remove_implicit_resolver(yaml.SafeLoader) path = Path(path) with open(path, "r", encoding="utf8") as file: return yaml.load(file, Loader=yaml.SafeLoader) ``` # model_config_pb2 Generated protocol buffer code. # packaging Pipeline packaging. This module contains classes and functionality for creating and validating pipeline configuration packages. ## `constants` Common constants used in 'packaging' module. ## `python_dependencies` Python dependencies This class handles specifying and validating Python dependencies. ### `PythonDependencies` Source code in `docs/industrial-ai-suite/sdk/simaticai/packaging/python_dependencies.py` ```python class PythonDependencies(): def __init__(self, python_version='3.11', dir: Union[str, os.PathLike] = None): """ This class handles Python dependencies Dependencies from remote repositories can be added via a requirements.txt file, or by calling the add_dependencies method. Dependencies can also be added as a single wheel or source distribution file, or as a collection in a zip archive via the add_python_packages method. The class can be converted to string, which will contain the dependencies in PEP508 format. """ self.python_version = python_version self.dependencies = {} self.python_packages = Path(tempfile.mkdtemp(dir=dir, prefix="dependencies_")) self.temp = Path(tempfile.mkdtemp(dir=dir, prefix="temp_")) self.optimize_dependencies = True self.index_url = None self.extra_index = [] def __str__(self): """ PEP508 representation of the dependencies. """ result = "" if self.index_url is not None: result += '# Index URL\n' result += f"{self.index_url}\n" if len(self.extra_index) > 0: result += "# Extra index urls\n" for url in self.extra_index: result += f"{url}\n" result += "# Runtime dependencies\n" for spec in self.dependencies.values(): result += str(spec) + "\n" return result def __repr__(self): return self.__str__() def clear(self): self.dependencies.clear() self.extra_index = [] self.index_url = None shutil.rmtree(self.python_packages, ignore_errors=True) shutil.rmtree(self.temp, ignore_errors=True) self.python_packages.mkdir(mode=0o700, exist_ok=True) self.temp.mkdir(mode=0o700, exist_ok=True) _logger.warning("Previously added dependencies have been removed.") def set_requirements(self, requirements_path: Union[str, os.PathLike]): self.clear() if Path(requirements_path).suffix == '.toml': dependencies, extra_index, index_url = pep508.parse_pyproject_toml(requirements_path) else: dependencies, extra_index, index_url = pep508.parse_requirements(requirements_path) for name, spec in dependencies.items(): _check_package_for_dependency_limitations(spec.name) if not any(spec.name.lower() == dep.lower() for dep in self.dependencies): self.dependencies[name] = spec _logger.info(f"Runtime dependency added: {spec}") else: _logger.warning(f"Dependency already exists: {spec}") if index_url is not None: self.index_url = index_url _logger.info(f"Index url added: {index_url}") for url in extra_index: self.extra_index.append(url) _logger.info(f"Extra index url added: {url}") def add_dependencies(self, packages: list): for package in packages: if isinstance(package, tuple): name, version = package spec = pep508.parse_line(f"{name}=={version}") else: spec = pep508.parse_line(f"{package}") _check_package_for_dependency_limitations(spec.name) if not any(spec.name.lower() == dep.lower() for dep in self.dependencies): self.dependencies[spec.name] = spec _logger.info(f"Runtime dependency added: {spec}") else: _logger.warning(f"Dependency already exists: {spec}") def add_python_packages(self, path: Union[str, os.PathLike]) -> None: path: Path = Path(path) if not path.is_file(): raise AssertionError(f"The file must be available on path {path.resolve()}") specs = [] tmp = None if is_wheel_file(path): name, version = get_wheel_name_version(path) specs.append((name, version, path)) elif is_pure_python_source(path): name, version = get_sdist_name_version(path) specs.append((name, version, path)) elif zipfile.is_zipfile(path): tmp = Path(tempfile.mkdtemp(dir=self.temp)) zip = zipfile.ZipFile(path) for pkg in zip.namelist(): zip.extract(pkg, path=tmp) file = tmp / Path(pkg).name if is_wheel_file(file): name, version = get_wheel_name_version(file) specs.append((name, version, file)) elif is_pure_python_source(file): name, version = get_sdist_name_version(file) specs.append((name, version, file)) else: _logger.warning(f"File skipped because it is not a wheel or pure python source: {pkg}") else: _logger.warning(f"File skipped because it is not a wheel or pure python source or a zip file: {path}") for name, version, path in specs: if name is not None: if version is None: spec = pep508.parse_line(f"{name}") else: spec = pep508.parse_line(f"{name}=={version}") _check_package_for_dependency_limitations(spec.name) if not any(spec.name.lower() == dep.lower() for dep in self.dependencies): shutil.copy(path, self.python_packages) self.dependencies[spec.name] = spec _logger.info(f"Runtime dependency added: {spec}") else: _logger.warning(f"Dependency already exists: {spec}") if tmp is not None: shutil.rmtree(tmp, ignore_errors=True) def _download_or_copy_dependency(self, name, version): dependency_url = urllib.parse.urlparse(version) # raises ValueError if the url is invalid dependency_path = Path(urllib.parse.unquote(dependency_url.path)) filename = dependency_path.name if "file" == dependency_url.scheme: # Possible Exceptions here: FileNotFoundError, PermissionError, OSError, IsADirectoryError, SameFileError if not dependency_path.is_file(): raise FileNotFoundError(f"The dependency '{name}' can not be found on path '{dependency_path}'") if (self.python_packages / filename).is_file(): _logger.warning(f"Dependency '{name}' will not be copied because it already exists in '{self.python_packages}' folder.") else: _logger.info(f"Dependency '{name}' will be copied to '{self.python_packages}' folder.") shutil.copy(dependency_path.resolve(), self.python_packages) else: # Possible Exceptions here: requests.exceptions.RequestException _logger.info(f"Dependency '{name}@{version}' will be downloaded from the repository.") response = requests.get(version) response.raise_for_status() with open(self.python_packages / filename, "wb") as f: f.write(response.content) return self.python_packages / filename def save(self, folder_path): # Downloads dependencies specified with url from remote repositories or copies them from local file system # Does not work with packages in source distribution format for name, dependency in self.dependencies.copy().items(): if isinstance(dependency.version, str): try: path = self._download_or_copy_dependency(name, dependency.version) _, version = get_wheel_name_version(path) self.dependencies[name] = pep508.parse_line(f"{name}=={version}") except requests.exceptions.RequestException as request_error: raise RuntimeError(f"Failed to download dependency '{dependency.name}=={dependency.version}' from the repository.") from request_error requirements_file_path = folder_path / REQUIREMENTS_TXT with open(requirements_file_path, "w") as f: f.write(str(self)) shutil.make_archive( base_name = folder_path / PYTHON_PACKAGES, root_dir = self.python_packages, format = 'zip', verbose = True, logger = _logger) def _check_if_index_url_is_set_to_pytorch_cpu(self): if self.index_url is None: return False if self.index_url.strip().startswith("--index-url") and _PYTORCH_CPU_REPO_URL in self.index_url: return True return False def enable_dependency_optimization(self): self.optimize_dependencies = True def disable_dependency_optimization(self): self.optimize_dependencies = False def validate(self): for spec in self.dependencies.values(): _check_package_for_dependency_limitations(spec.name) found_gpu_dependency = any(dep in _GPU_DEPENDENCIES for dep in self.dependencies) if not found_gpu_dependency: return if self.optimize_dependencies: if self.index_url is None: self.index_url = f"--index-url {_PYTORCH_CPU_REPO_URL}" added_pypi_warning = "" if not any([_PYPI_REPO_URL in item for item in self.extra_index]): self.extra_index.insert(0, f"--extra-index-url {_PYPI_REPO_URL}") added_pypi_warning = ADDED_PYPI_WARNING_MSG _logger.warning(f"WARNING! {REPO_MODIFICATION_WARNING_MSG} {added_pypi_warning}") elif self._check_if_index_url_is_set_to_pytorch_cpu(): if not any([_PYPI_REPO_URL in item for item in self.extra_index]): self.extra_index.insert(0, f"--extra-index-url {_PYPI_REPO_URL}") _logger.warning(f"WARNING! {REPO_MODIFICATION_WARNING_MSG} {ADDED_PYPI_WARNING_MSG}") else: user_defined_index_url = self.index_url.replace("--index-url", "--extra-index-url", 1) self.index_url = f"--index-url {_PYTORCH_CPU_REPO_URL}" self.extra_index.insert(0, user_defined_index_url) _logger.warning(f"WARNING! {REPO_MODIFICATION_WARNING_MSG} {INDEX_URL_MOVED_WARNING_MSG}") else: if not self._check_if_index_url_is_set_to_pytorch_cpu(): _logger.warning( "WARNING! The resulting package could contain unused GPU dependencies " "which considerably increase the file size.") ``` #### `__init__(python_version='3.11', dir=None)` This class handles Python dependencies Dependencies from remote repositories can be added via a requirements.txt file, or by calling the add_dependencies method. Dependencies can also be added as a single wheel or source distribution file, or as a collection in a zip archive via the add_python_packages method. The class can be converted to string, which will contain the dependencies in PEP508 format. Source code in `docs/industrial-ai-suite/sdk/simaticai/packaging/python_dependencies.py` ```python def __init__(self, python_version='3.11', dir: Union[str, os.PathLike] = None): """ This class handles Python dependencies Dependencies from remote repositories can be added via a requirements.txt file, or by calling the add_dependencies method. Dependencies can also be added as a single wheel or source distribution file, or as a collection in a zip archive via the add_python_packages method. The class can be converted to string, which will contain the dependencies in PEP508 format. """ self.python_version = python_version self.dependencies = {} self.python_packages = Path(tempfile.mkdtemp(dir=dir, prefix="dependencies_")) self.temp = Path(tempfile.mkdtemp(dir=dir, prefix="temp_")) self.optimize_dependencies = True self.index_url = None self.extra_index = [] ``` #### `__str__()` PEP508 representation of the dependencies. Source code in `docs/industrial-ai-suite/sdk/simaticai/packaging/python_dependencies.py` ```python def __str__(self): """ PEP508 representation of the dependencies. """ result = "" if self.index_url is not None: result += '# Index URL\n' result += f"{self.index_url}\n" if len(self.extra_index) > 0: result += "# Extra index urls\n" for url in self.extra_index: result += f"{url}\n" result += "# Runtime dependencies\n" for spec in self.dependencies.values(): result += str(spec) + "\n" return result ``` ## `wheelhouse` Methods for downloading and validating dependencies This module collects all the necessary methods for downloading wheel or source distributions, and validation methods for checking if the whole collection could be installed in the AI Inference Server's Python runtime environment. ### `assert_none_parameters(**kwargs)` Checks if any of the given parameters are None. Returns: | Type | Description | | ------ | ----------------------------------- | | `bool` | True if all parameters are not None | Source code in `docs/industrial-ai-suite/sdk/simaticai/packaging/wheelhouse.py` ```python def assert_none_parameters(**kwargs) -> bool: """ Checks if any of the given parameters are None. Returns: True if all parameters are not None Raises: AssertionError: otherwise. """ none_values = [k for k, v in kwargs.items() if v is None] if 0 < len(none_values): none_values = ", ".join(none_values) raise AssertionError(f"Parameters can not be None: {none_values}") return True ``` ### `is_wheel_file(path)` Checks whether the file on the given `path` is a wheel file. Parameters: | Name | Type | Description | Default | | ------ | ------------- | ------------------------------------------------ | ---------- | | `path` | `path - like` | The relative or absolute path of the wheel file. | *required* | Source code in `docs/industrial-ai-suite/sdk/simaticai/packaging/wheelhouse.py` ```python def is_wheel_file(path: os.PathLike) -> bool: """ Checks whether the file on the given `path` is a wheel file. Args: path (path-like): The relative or absolute path of the wheel file. Returns: bool: True if the zipfile contains a WHEEL text file, False otherwise. """ if zipfile.is_zipfile(path): _wheel = zipfile.ZipFile(path) return 'WHEEL' in [f.split("/")[-1] for f in _wheel.namelist()] return False ``` ### `is_source_file(path)` Checks whether the file on the given `path` is a python source distribtion file. Parameters: | Name | Type | Description | Default | | ------ | ------------- | ---------------------------------------------------------------- | ---------- | | `path` | `path - like` | The relative or absolute path of the zip or tar.gz archive file. | *required* | Source code in `docs/industrial-ai-suite/sdk/simaticai/packaging/wheelhouse.py` ```python def is_source_file(path: os.PathLike) -> bool: """ Checks whether the file on the given `path` is a python source distribtion file. Args: path (path-like): The relative or absolute path of the zip or tar.gz archive file. Returns: bool: True if the archive file contains a PKG-INFO text file, False otherwise. """ if zipfile.is_zipfile(path): _archive = zipfile.ZipFile(path) return 'PKG-INFO' in [f.split("/")[-1] for f in _archive.namelist()] if tarfile.is_tarfile(path): with tarfile.open(path) as _archive: return 'PKG-INFO' in [f.split("/")[-1] for f in _archive.getnames()] return False ``` ### `is_pure_python_source(archive_path)` Checks whether the given source distribution contains only Python sources. This method handles source distributions in a unified way. It searches for 'PKG-INFO' files, collects the programming languages used in the source, and returns True if only the Python language was used. Parameters: | Name | Type | Description | Default | | -------------- | ------------- | ---------------------------------------------------------------- | ---------- | | `archive_path` | `path - like` | The relative or absolute path of the zip or tar.gz archive file. | *required* | Source code in `docs/industrial-ai-suite/sdk/simaticai/packaging/wheelhouse.py` ```python def is_pure_python_source(archive_path: Union[str, os.PathLike]): """ Checks whether the given source distribution contains only Python sources. This method handles source distributions in a unified way. It searches for 'PKG-INFO' files, collects the programming languages used in the source, and returns True if only the Python language was used. Args: archive_path (path-like): The relative or absolute path of the zip or tar.gz archive file. Returns: bool: True if the archive file contains only Python sources. """ headers = _extract_pkg_info(archive_path) if headers is None: return False classifiers = map(lambda header: header.get_all("classifier"), headers) classifiers = map(lambda classifier: [] if classifier is None else classifier, classifiers) classifiers = chain.from_iterable(classifiers) programming_languages = filter(lambda line: line.startswith('Programming Language ::'), classifiers) programming_languages = map(lambda line: line.split("::")[1].strip().lower(), programming_languages) return all(map(lambda txt: txt == 'python', programming_languages)) ``` ### `get_wheel_name_version(archive_path)` Extracts the package name and version from a wheel file. Parameters: | Name | Type | Description | Default | | -------------- | ------------- | -------------------------------------------------------- | ---------- | | `archive_path` | `path - like` | The relative or absolute path of the wheel archive file. | *required* | Source code in `docs/industrial-ai-suite/sdk/simaticai/packaging/wheelhouse.py` ```python def get_wheel_name_version(archive_path: Union[str, os.PathLike]): """ Extracts the package name and version from a wheel file. Args: archive_path (path-like): The relative or absolute path of the wheel archive file. Returns: (str, str): The name and version of the wheel if successful, (None,None) otherwise. """ with zipfile.ZipFile(archive_path, "r") as archive: files = archive.namelist() METADATA = list(filter(lambda filepath: filepath.endswith('METADATA'), files)) if 0 < len(METADATA): headers = map( lambda filename: archive.read(filename).decode("utf-8"), METADATA) headers = map(lambda txt: Parser().parsestr(text=txt, headersonly=True), headers) headers = list(headers)[0] name = headers.get("name") version = headers.get("version") return (name, version) return (None, None) ``` ### `get_sdist_name_version(archive_path)` Extracts the package name and version from a source distribution file. Parameters: | Name | Type | Description | Default | | -------------- | ------------- | ---------------------------------------------------------------- | ---------- | | `archive_path` | `path - like` | The relative or absolute path of the zip or tar.gz archive file. | *required* | Source code in `docs/industrial-ai-suite/sdk/simaticai/packaging/wheelhouse.py` ```python def get_sdist_name_version(archive_path: Union[str, os.PathLike]): """ Extracts the package name and version from a source distribution file. Args: archive_path (path-like): The relative or absolute path of the zip or tar.gz archive file. Returns: (str, str): The name and version of the source distribution if successful, (None,None) otherwise. """ headers = _extract_pkg_info(archive_path) if headers is None: return (None, None) headers = headers[0] name = headers.get("name") version = headers.get("version") return (name, version) ``` ### `check_directory_has_only_wheels_and_pure_sdist(python_packages_folder)` Checks all files in a directory if they are wheel files or pure Python source distributions. Parameters: | Name | Type | Description | Default | | ------------------------ | ------------- | ------------------------------------------------------------- | ---------- | | `python_packages_folder` | `path - like` | The relative or absolute path of the directory to be checked. | *required* | Source code in `docs/industrial-ai-suite/sdk/simaticai/packaging/wheelhouse.py` ```python def check_directory_has_only_wheels_and_pure_sdist(python_packages_folder: Union[str, os.PathLike]): """ Checks all files in a directory if they are wheel files or pure Python source distributions. Args: python_packages_folder (path-like): The relative or absolute path of the directory to be checked. Raises: AssertionError: If the directory contains other files than wheels or pure Python source distributions. """ not_pure = [] for file in list(python_packages_folder.iterdir()): if not (is_wheel_file(file) or is_pure_python_source(file)): not_pure.append(file.name) if 0 < len(not_pure): not_pure = "\n".join(not_pure) raise AssertionError(dedent(f""" One or more source dependencies are not pure Python sources. You need to convert them to wheel files for the target platform manually. List of not pure Python source distributions: {not_pure} """)) ``` # payloads ## `Connection` Represents a connection with a name and an optional predefined combination of connection type and payload format. Attributes: | Name | Type | Description | | -------- | ------------------------------------------ | --------------------------------------------------------------- | | `name` | `str` | The name of the connection. | | `cptype` | `Optional[ConnectionTypeAndPayloadFormat]` | The connection type and payload format, represented as an Enum. | Methods: | Name | Description | | ------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `__setattr__` | Custom attribute setter that validates attribute assignment. - 'name': Accepts any value. - 'cptype': Must be None or an instance of ConnectionTypeAndPayloadFormat. - Any other attribute: Raises ValueError. | | `__dict__` | Returns a dictionary representation of the connection, including: - 'connectorName': The name of the connector. - 'connectorType': The type of the connector (if cptype is set). - 'payloadType': The payload type (if cptype is set). | Source code in `docs/industrial-ai-suite/sdk/simaticai/payloads/connection.py` ```python @dataclass class Connection: """ Represents a connection with a name and an optional predefined combination of connection type and payload format. Attributes: name (str): The name of the connection. cptype (Optional[ConnectionTypeAndPayloadFormat]): The connection type and payload format, represented as an Enum. Methods: __setattr__(name, value): Custom attribute setter that validates attribute assignment. - 'name': Accepts any value. - 'cptype': Must be None or an instance of ConnectionTypeAndPayloadFormat. - Any other attribute: Raises ValueError. __dict__(): Returns a dictionary representation of the connection, including: - 'connectorName': The name of the connector. - 'connectorType': The type of the connector (if cptype is set). - 'payloadType': The payload type (if cptype is set). """ name: str cptype: Optional[ConnectionTypeAndPayloadFormat] = None def __setattr__(self, name, value): """ Setter for all attributes """ match name: case "name": pass case "cptype": if value is not None and not isinstance(value, ConnectionTypeAndPayloadFormat): raise ValueError("ERROR! 'type' must be an instance of ConnectionTypeAndPayloadFormat Enum.") case _: raise ValueError(f"ERROR! Unknown attribute '{name}' for connector '{self.name}'.") super(self.__class__, self).__setattr__(name, value) def __dict__(self): connection_dict = { "connectorName": self.name } if self.cptype is not None: connection_dict["connectorType"] = self.cptype.value[0] connection_dict["payloadType"] = self.cptype.value[1] return connection_dict ``` ### `__setattr__(name, value)` Setter for all attributes Source code in `docs/industrial-ai-suite/sdk/simaticai/payloads/connection.py` ```python def __setattr__(self, name, value): """ Setter for all attributes """ match name: case "name": pass case "cptype": if value is not None and not isinstance(value, ConnectionTypeAndPayloadFormat): raise ValueError("ERROR! 'type' must be an instance of ConnectionTypeAndPayloadFormat Enum.") case _: raise ValueError(f"ERROR! Unknown attribute '{name}' for connector '{self.name}'.") super(self.__class__, self).__setattr__(name, value) ``` ## `ConnectionTypeAndPayloadFormat` Bases: `Enum` Enum for predefined connection on AI Inference Server. The Attributes of the enum are tuples with the Type and Payload Format supported on AI Inference Server. Attributes: | Name | Type | Description | | ----------------------------- | ------- | -------------------------------- | | `Databus_String` | `tuple` | ("databus", "String") | | `Databus_SIMATICv1` | `tuple` | ("databus", "S7Json") | | `External_Databus` | `tuple` | ("mqtt", "String") | | `IE_Vision` | `tuple` | ("vision", "VisionConnectorRaw") | | `ZMQ_Vision` | `tuple` | ("zmq", "VisionConnectorRaw") | | `ZMQ_Multipart` | `tuple` | ("zmq", "ZmqPayload") | | `ZMQ_Multipart_with_Metadata` | `tuple` | ("zmq", "ZmqImageOutput") | | `Realtime_Backbone` | `tuple` | ("rib", "RIBPayload") | Source code in `docs/industrial-ai-suite/sdk/simaticai/payloads/connection.py` ```python class ConnectionTypeAndPayloadFormat(Enum): """ Enum for predefined connection on AI Inference Server. The Attributes of the enum are tuples with the Type and Payload Format supported on AI Inference Server. Attributes: Databus_String (tuple): ("databus", "String") Databus_SIMATICv1 (tuple): ("databus", "S7Json") External_Databus (tuple): ("mqtt", "String") IE_Vision (tuple): ("vision", "VisionConnectorRaw") ZMQ_Vision (tuple): ("zmq", "VisionConnectorRaw") ZMQ_Multipart (tuple): ("zmq", "ZmqPayload") ZMQ_Multipart_with_Metadata (tuple): ("zmq", "ZmqImageOutput") Realtime_Backbone (tuple): ("rib", "RIBPayload") """ Databus_String = ("databus", "String") """ Databus String or Json """ Databus_SIMATICv1 = ("databus", "S7Json") """ SIMATIC v1 (S7, S7+, OPC-UA, PROFINET IO) """ External_String = ("mqtt", "String") """ External Databus String """ IE_Vision = ("vision", "VisionConnectorRaw") """ IE Vision Connector (High throughput, ZMQ Based, Multipart) """ ZMQ_Vision = ("zmq", "VisionConnectorRaw") """ Zero Message Queue Vision Connector Raw (Vision payload) """ ZMQ_Multipart = ("zmq", "ZmqPayload") """ Zero Message Queue Internal multipart message (Topic, binary data or string) """ ZMQ_Multipart_with_Metadata = ("zmq", "ZmqImageOutput") """ Zero Message Queue Internal multipart message (Topic, metadata, binary) """ Realtime_Backbone = ("rib", "RIBPayload") """ Real-time Information Backbone """ ``` ### `Databus_String = ('databus', 'String')` Databus String or Json ### `Databus_SIMATICv1 = ('databus', 'S7Json')` SIMATIC v1 (S7, S7+, OPC-UA, PROFINET IO) ### `External_String = ('mqtt', 'String')` External Databus String ### `IE_Vision = ('vision', 'VisionConnectorRaw')` IE Vision Connector (High throughput, ZMQ Based, Multipart) ### `ZMQ_Vision = ('zmq', 'VisionConnectorRaw')` Zero Message Queue Vision Connector Raw (Vision payload) ### `ZMQ_Multipart = ('zmq', 'ZmqPayload')` Zero Message Queue Internal multipart message (Topic, binary data or string) ### `ZMQ_Multipart_with_Metadata = ('zmq', 'ZmqImageOutput')` Zero Message Queue Internal multipart message (Topic, metadata, binary) ### `Realtime_Backbone = ('rib', 'RIBPayload')` Real-time Information Backbone ## `PipelineVariable` Bases: `MappableMixin` Represents a variable in a pipeline, linking a component's variable to a connection and optional mapping. For IE Vision connections, cameraName should be defined, otherwise tagName is preferred. Attributes: | Name | Type | Description | | --------------- | ---------------------- | -------------------------------------------------------------- | | `componentName` | `str` | The name of the component the variable belongs to. | | `variableName` | `str` | The name of the variable. | | `connection` | `Optional[Connection]` | The connection associated with the variable. | | `tagName` | `Optional[str]` | The tag name to map to. | | `cameraName` | `Optional[str]` | The camera name to map to. Used only via IE_Vision connection. | | `mapping` | `Optional[str]` | The topic or camera id to map to. | Source code in `docs/industrial-ai-suite/sdk/simaticai/payloads/pipeline_variable.py` ```python @dataclass class PipelineVariable(MappableMixin): """ Represents a variable in a pipeline, linking a component's variable to a connection and optional mapping. For IE Vision connections, cameraName should be defined, otherwise tagName is preferred. Attributes: componentName (str): The name of the component the variable belongs to. variableName (str): The name of the variable. connection (Optional[Connection]): The connection associated with the variable. tagName (Optional[str]): The tag name to map to. cameraName (Optional[str]): The camera name to map to. Used only via IE_Vision connection. mapping (Optional[str]): The topic or camera id to map to. """ componentName: str variableName: str # Mixin fields as dataclass fields connection: Optional[Connection] = field(default=None) tagName: Optional[str] = field(default=None) cameraName: Optional[str] = field(default=None) mapping: Optional[str] = field(default=None) def check_connection_type_compatibility(self, variable_type: str) -> tuple[list, list]: if self.connection is None: return [], [] supported_types = SUPPORTED_TYPES_BY_CONNECTION.get(self.connection.cptype, []) flags = type_validation_flags(variable_type, supported_types) if self.connection.cptype == ConnectionTypeAndPayloadFormat.Realtime_Backbone: warning_flags = [] error_flags = [f.value for f in flags] else: warning_flags = [f.value for f in flags if f != TypeValidationFlags.ARRAY_WITH_MISSING_SIZE] error_flags = [] return warning_flags, error_flags def __getitem__(self, item): return getattr(self, item) def __setattr__(self, name, value): """ Setter for all attributes """ match name: case "componentName": if re.match("^[a-zA-Z0-9_-]+$", value) is None: raise ValueError(f"ERROR! Invalid name '{value}'. Only alphanumeric characters and underscores are allowed.") case "variableName": if re.match("^[a-zA-Z0-9_-]+$", value) is None: raise ValueError(f"ERROR! Invalid name '{value}'. Only alphanumeric characters and underscores are allowed.") if value in reserved_names: raise ValueError(f"ERROR! '{value}' is a reserved name and cannot be used as variable name.") case _: # Let the mixin handle mapping attributes pass super().__setattr__(name, value) def __dict__(self, component_io_list: list) -> dict: json_dict = { "name": self.variableName, "type": component_io_list[self.variableName]['type'] } json_dict.update(self.__mappable_dict__()) return json_dict ``` ### `__setattr__(name, value)` Setter for all attributes Source code in `docs/industrial-ai-suite/sdk/simaticai/payloads/pipeline_variable.py` ```python def __setattr__(self, name, value): """ Setter for all attributes """ match name: case "componentName": if re.match("^[a-zA-Z0-9_-]+$", value) is None: raise ValueError(f"ERROR! Invalid name '{value}'. Only alphanumeric characters and underscores are allowed.") case "variableName": if re.match("^[a-zA-Z0-9_-]+$", value) is None: raise ValueError(f"ERROR! Invalid name '{value}'. Only alphanumeric characters and underscores are allowed.") if value in reserved_names: raise ValueError(f"ERROR! '{value}' is a reserved name and cannot be used as variable name.") case _: # Let the mixin handle mapping attributes pass super().__setattr__(name, value) ``` ## `PipelineParameter` Bases: `MappableMixin` Source code in `docs/industrial-ai-suite/sdk/simaticai/payloads/pipeline_variable.py` ```python @dataclass class PipelineParameter(MappableMixin): name: str defaultValue: Optional[str] = None dtype: Optional[str] = None # 'String', 'Integer', 'Float', 'Boolean' description: Optional[str] = None topicBased: Optional[bool] = False valueTopic: Optional[str] = None # Mixin fields as dataclass fields connection: Optional[Connection] = field(default=None) tagName: Optional[str] = field(default=None) mapping: Optional[str] = field(default=None) def __post_init__(self): # Validate if the topicBased flag is False, then valueTopic, connection, tag, and topic should not be set if not self.topicBased and (self.valueTopic is not None or self.connection is not None or self.tagName is not None or self.mapping is not None): raise ValueError("ERROR! 'valueTopic', 'connection', 'tag', and 'topic' must be None when 'topicBased' is False.") if self.dtype != _get_is_type(type(self.defaultValue)): raise ValueError(f"'The given value type does not match the type of defaultValue: '{self.defaultValue}'") def __setattr__(self, name, value): """ Setter for all attributes """ match name: case "name": _check_name_validity(value) case "dtype": if value is None: return elif value not in ['String', 'Integer', 'Double', 'Boolean']: raise ValueError("ERROR! 'dtype' must be one of 'String', 'Integer', 'Double', 'Boolean', or None.") case "defaultValue": dtype = _get_is_type(type(value)) if dtype is None: raise ValueError("ERROR! 'dtype' must be one of 'String', 'Integer', 'Double', 'Boolean', or None.") self.dtype = dtype case "description": _check_description_validity(value) case "topicBased": if not isinstance(value, bool): raise ValueError("Type of the given `topic_based` parameter is not `bool`") case "valueTopic": if not (value is None or value == ""): self.topicBased = True case _: # Let the mixin handle mapping attributes pass super().__setattr__(name, value) def has_valid_connection(self) -> set: if self.connection is None: return True supported_types = SUPPORTED_TYPES_BY_CONNECTION.get(self.connection.cptype, []) return type_validation_flags(self.dtype, supported_types) == set() def __param_dict__(self) -> dict: json_dict = { 'name': self.name, 'defaultValue': self.defaultValue, 'dtype': self.dtype } json_dict.update(self.__mappable_dict__()) json_dict['type'] = json_dict.pop('dtype', None) if self.topicBased: json_dict['topicBased'] = True json_dict['valueTopic'] = json_dict.pop('topic', None) return json_dict ``` ### `__setattr__(name, value)` Setter for all attributes Source code in `docs/industrial-ai-suite/sdk/simaticai/payloads/pipeline_variable.py` ```python def __setattr__(self, name, value): """ Setter for all attributes """ match name: case "name": _check_name_validity(value) case "dtype": if value is None: return elif value not in ['String', 'Integer', 'Double', 'Boolean']: raise ValueError("ERROR! 'dtype' must be one of 'String', 'Integer', 'Double', 'Boolean', or None.") case "defaultValue": dtype = _get_is_type(type(value)) if dtype is None: raise ValueError("ERROR! 'dtype' must be one of 'String', 'Integer', 'Double', 'Boolean', or None.") self.dtype = dtype case "description": _check_description_validity(value) case "topicBased": if not isinstance(value, bool): raise ValueError("Type of the given `topic_based` parameter is not `bool`") case "valueTopic": if not (value is None or value == ""): self.topicBased = True case _: # Let the mixin handle mapping attributes pass super().__setattr__(name, value) ``` ## `SecretPipelineParameter` Represents a secret pipeline parameter for handling sensitive information. Cannot be modified after creation. Attributes: | Name | Type | Description | | -------------- | --------------- | ---------------------------------------------------------------------------------------------- | | `name` | `str` | The name of the secret parameter. | | `description` | `Optional[str]` | An optional description of the secret parameter. | | `defaultValue` | `str` | Pipeline parameters must have a default value, which is an empty string for secret parameters. | | `dtype` | `str` | The data type of the secret parameter is always "Secret". | Source code in `docs/industrial-ai-suite/sdk/simaticai/payloads/pipeline_variable.py` ```python @dataclass(frozen=True) class SecretPipelineParameter: """ Represents a secret pipeline parameter for handling sensitive information. Cannot be modified after creation. Attributes: name (str): The name of the secret parameter. description (Optional[str]): An optional description of the secret parameter. defaultValue (str): Pipeline parameters must have a default value, which is an empty string for secret parameters. dtype (str): The data type of the secret parameter is always "Secret". """ name: str description: Optional[str] = None defaultValue: str = field(init=False, default="") dtype: str = field(init=False, default="Secret") def __post_init__(self): _check_name_validity(self.name) _check_description_validity(self.description) def __param_dict__(self) -> dict: json_dict = { 'name': self.name, 'defaultValue': "", 'type': self.dtype } return json_dict ``` ## `ConnectionTypeAndPayloadFormat` Bases: `Enum` Enum for predefined connection on AI Inference Server. The Attributes of the enum are tuples with the Type and Payload Format supported on AI Inference Server. Attributes: | Name | Type | Description | | ----------------------------- | ------- | -------------------------------- | | `Databus_String` | `tuple` | ("databus", "String") | | `Databus_SIMATICv1` | `tuple` | ("databus", "S7Json") | | `External_Databus` | `tuple` | ("mqtt", "String") | | `IE_Vision` | `tuple` | ("vision", "VisionConnectorRaw") | | `ZMQ_Vision` | `tuple` | ("zmq", "VisionConnectorRaw") | | `ZMQ_Multipart` | `tuple` | ("zmq", "ZmqPayload") | | `ZMQ_Multipart_with_Metadata` | `tuple` | ("zmq", "ZmqImageOutput") | | `Realtime_Backbone` | `tuple` | ("rib", "RIBPayload") | Source code in `docs/industrial-ai-suite/sdk/simaticai/payloads/connection.py` ```python class ConnectionTypeAndPayloadFormat(Enum): """ Enum for predefined connection on AI Inference Server. The Attributes of the enum are tuples with the Type and Payload Format supported on AI Inference Server. Attributes: Databus_String (tuple): ("databus", "String") Databus_SIMATICv1 (tuple): ("databus", "S7Json") External_Databus (tuple): ("mqtt", "String") IE_Vision (tuple): ("vision", "VisionConnectorRaw") ZMQ_Vision (tuple): ("zmq", "VisionConnectorRaw") ZMQ_Multipart (tuple): ("zmq", "ZmqPayload") ZMQ_Multipart_with_Metadata (tuple): ("zmq", "ZmqImageOutput") Realtime_Backbone (tuple): ("rib", "RIBPayload") """ Databus_String = ("databus", "String") """ Databus String or Json """ Databus_SIMATICv1 = ("databus", "S7Json") """ SIMATIC v1 (S7, S7+, OPC-UA, PROFINET IO) """ External_String = ("mqtt", "String") """ External Databus String """ IE_Vision = ("vision", "VisionConnectorRaw") """ IE Vision Connector (High throughput, ZMQ Based, Multipart) """ ZMQ_Vision = ("zmq", "VisionConnectorRaw") """ Zero Message Queue Vision Connector Raw (Vision payload) """ ZMQ_Multipart = ("zmq", "ZmqPayload") """ Zero Message Queue Internal multipart message (Topic, binary data or string) """ ZMQ_Multipart_with_Metadata = ("zmq", "ZmqImageOutput") """ Zero Message Queue Internal multipart message (Topic, metadata, binary) """ Realtime_Backbone = ("rib", "RIBPayload") """ Real-time Information Backbone """ ``` ### `Databus_String = ('databus', 'String')` Databus String or Json ### `Databus_SIMATICv1 = ('databus', 'S7Json')` SIMATIC v1 (S7, S7+, OPC-UA, PROFINET IO) ### `External_String = ('mqtt', 'String')` External Databus String ### `IE_Vision = ('vision', 'VisionConnectorRaw')` IE Vision Connector (High throughput, ZMQ Based, Multipart) ### `ZMQ_Vision = ('zmq', 'VisionConnectorRaw')` Zero Message Queue Vision Connector Raw (Vision payload) ### `ZMQ_Multipart = ('zmq', 'ZmqPayload')` Zero Message Queue Internal multipart message (Topic, binary data or string) ### `ZMQ_Multipart_with_Metadata = ('zmq', 'ZmqImageOutput')` Zero Message Queue Internal multipart message (Topic, metadata, binary) ### `Realtime_Backbone = ('rib', 'RIBPayload')` Real-time Information Backbone ## `Connection` Represents a connection with a name and an optional predefined combination of connection type and payload format. Attributes: | Name | Type | Description | | -------- | ------------------------------------------ | --------------------------------------------------------------- | | `name` | `str` | The name of the connection. | | `cptype` | `Optional[ConnectionTypeAndPayloadFormat]` | The connection type and payload format, represented as an Enum. | Methods: | Name | Description | | ------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `__setattr__` | Custom attribute setter that validates attribute assignment. - 'name': Accepts any value. - 'cptype': Must be None or an instance of ConnectionTypeAndPayloadFormat. - Any other attribute: Raises ValueError. | | `__dict__` | Returns a dictionary representation of the connection, including: - 'connectorName': The name of the connector. - 'connectorType': The type of the connector (if cptype is set). - 'payloadType': The payload type (if cptype is set). | Source code in `docs/industrial-ai-suite/sdk/simaticai/payloads/connection.py` ```python @dataclass class Connection: """ Represents a connection with a name and an optional predefined combination of connection type and payload format. Attributes: name (str): The name of the connection. cptype (Optional[ConnectionTypeAndPayloadFormat]): The connection type and payload format, represented as an Enum. Methods: __setattr__(name, value): Custom attribute setter that validates attribute assignment. - 'name': Accepts any value. - 'cptype': Must be None or an instance of ConnectionTypeAndPayloadFormat. - Any other attribute: Raises ValueError. __dict__(): Returns a dictionary representation of the connection, including: - 'connectorName': The name of the connector. - 'connectorType': The type of the connector (if cptype is set). - 'payloadType': The payload type (if cptype is set). """ name: str cptype: Optional[ConnectionTypeAndPayloadFormat] = None def __setattr__(self, name, value): """ Setter for all attributes """ match name: case "name": pass case "cptype": if value is not None and not isinstance(value, ConnectionTypeAndPayloadFormat): raise ValueError("ERROR! 'type' must be an instance of ConnectionTypeAndPayloadFormat Enum.") case _: raise ValueError(f"ERROR! Unknown attribute '{name}' for connector '{self.name}'.") super(self.__class__, self).__setattr__(name, value) def __dict__(self): connection_dict = { "connectorName": self.name } if self.cptype is not None: connection_dict["connectorType"] = self.cptype.value[0] connection_dict["payloadType"] = self.cptype.value[1] return connection_dict ``` ### `__setattr__(name, value)` Setter for all attributes Source code in `docs/industrial-ai-suite/sdk/simaticai/payloads/connection.py` ```python def __setattr__(self, name, value): """ Setter for all attributes """ match name: case "name": pass case "cptype": if value is not None and not isinstance(value, ConnectionTypeAndPayloadFormat): raise ValueError("ERROR! 'type' must be an instance of ConnectionTypeAndPayloadFormat Enum.") case _: raise ValueError(f"ERROR! Unknown attribute '{name}' for connector '{self.name}'.") super(self.__class__, self).__setattr__(name, value) ``` This module defines the ImageDetails and ImageSet data structures for handling image payloads in the context of AI Inference Server communication. It includes functionality for parsing and formatting timestamps, converting image formats, and serializing/deserializing image data. Examples: - Creating an ImageDetails object from an image file: image_details = ImageDetails.from_image("path/to/image.jpg", format="RGB8") - Creating an ImageSet and adding images: image_set = ImageSet() image_set.add_image(image_details) images_set.add_image(ImageDetails.from_image("path/to/another_image.png", format="Mono8")) - Converting a BGR image to a different format: converted_image, height, width = ImageDetails.convert_image_from_BGR(bgr_image, "Mono8") - Creating an ImageSet from a dictionary payload: image_set = ImageSet.from_dict(payload_dict) - Serializing an ImageSet to a dictionary: payload_dict = image_set.to_dict() Vision Payload Specification However, the above documentation is incomplete. The following are the Required fields for ImageSet, based on AI Inference Server source {"version", FieldType::String}, {"cameraid", FieldType::String}, {"timestamp", FieldType::String}, {"detail", FieldType::StructArray} }; Required fields for ImageDetails, based on AI Inference Server source: {"id", FieldType::String}, {"seq", FieldType::Integer}, {"format", FieldType::String}, {"image", FieldType::Binary} }; ## `ImageFormat` Bases: `Enum` Enumeration of supported image formats. To extend supported formats, update the ImageDetails conversion methods accordingly. Source code in `docs/industrial-ai-suite/sdk/simaticai/payloads/imageset.py` ```python class ImageFormat(Enum): """Enumeration of supported image formats. To extend supported formats, update the ImageDetails conversion methods accordingly. """ RGB8 = "RGB8" BGR8 = "BGR8" YUV422Packed = "YUV422Packed" YUV422_YUYV_Packed = "YUV422_YUYV_Packed" Mono8 = "Mono8" BayerRG8 = "BayerRG8" BayerGR8 = "BayerGR8" BayerBG8 = "BayerBG8" BayerGB8 = "BayerGB8" ``` ## `ImageDetails` Data structure representing the details of a single image in an ImageSet payload. Represents the details of a single image in an ImageSet payload. Fields: id (str): Unique identifier for the image. width (int): Width of the image in pixels. height (int): Height of the image in pixels. seq (int): Sequence number of the image. timestamp (datetime): Timestamp when the image was captured. metadata (dict): Additional metadata associated with the image. format (ImageFormat): Format of the image. image (bytes): Binary data of the image. Source code in `docs/industrial-ai-suite/sdk/simaticai/payloads/imageset.py` ```python @dataclass class ImageDetails: """Data structure representing the details of a single image in an ImageSet payload. Represents the details of a single image in an ImageSet payload. Fields: id (str): Unique identifier for the image. width (int): Width of the image in pixels. height (int): Height of the image in pixels. seq (int): Sequence number of the image. timestamp (datetime): Timestamp when the image was captured. metadata (dict): Additional metadata associated with the image. format (ImageFormat): Format of the image. image (bytes): Binary data of the image. """ id: str width: int height: int seq: int = field(default=0) timestamp: datetime = field(default_factory=lambda: datetime.now(timezone.utc)) metadata: dict = field(default_factory=dict) format: ImageFormat = field(default=ImageFormat.RGB8) image: bytes = field(default=b"") def __post_init__(self): assert isinstance(self.id, str) assert isinstance(self.timestamp, datetime) assert isinstance(self.width, int) and self.width > 0 assert isinstance(self.height, int) and self.height > 0 assert isinstance(self.format, ImageFormat) and self.format in ImageFormat.__members__.values() assert isinstance(self.image, bytes) and len(self.image) > 0 @staticmethod def _parse_image_format(format: str | ImageFormat | None = None) -> ImageFormat: match format: case None: return ImageFormat.RGB8 case ImageFormat(): return format case str() if format in ImageFormat.__members__.keys(): return ImageFormat[format] _supported_image_formats = [format.value for format in ImageFormat.__members__.values()] raise ValueError(f"ERROR Provided image format '{format}' is not supported. image_format must be one of {_supported_image_formats}") @staticmethod def from_image(image_path: Path | str, format: ImageFormat | str | None = None) -> 'ImageDetails': """Creates an ImageDetails object from an image file. Args: image_path (Path | str): Path to the image file. format (ImageFormat | str | None): Desired image format (default: RGB8). Raises: ValueError: If the image file is not found or cannot be read. Returns: ImageDetails: Created ImageDetails object. """ format = ImageDetails._parse_image_format(format) image_path = Path(image_path) image = cv2.imread(str(image_path)) if image is None: raise ValueError(f"Image file {image_path} not found") res_image, height, width = ImageDetails.convert_image_from_BGR(image, format) return ImageDetails( id=image_path.name, timestamp=datetime.now(timezone.utc), width=width, height=height, format=format, image=res_image ) @staticmethod def _bgr_to_bayer(bgr_image: np.ndarray, bayer_order: str) -> np.ndarray: (height, width) = bgr_image.shape[:2] bayer_image = np.zeros((height, width), dtype=np.uint8) BLUE, GREEN, RED = 0, 1, 2 match bayer_order: case 'BayerRG8': channels = (RED, GREEN, GREEN, BLUE) case 'BayerGR8': channels = (GREEN, RED, BLUE, GREEN) case 'BayerBG8': channels = (BLUE, GREEN, GREEN, RED) case 'BayerGB8': channels = (GREEN, BLUE, RED, GREEN) case _: raise ValueError(f"Unsupported bayer order: {bayer_order}") top_left_ch, top_right_ch, bottom_left_ch, bottom_right_ch = channels bayer_image[0::2, 0::2] = bgr_image[0::2, 0::2, top_left_ch] bayer_image[0::2, 1::2] = bgr_image[0::2, 1::2, top_right_ch] bayer_image[1::2, 0::2] = bgr_image[1::2, 0::2, bottom_left_ch] bayer_image[1::2, 1::2] = bgr_image[1::2, 1::2, bottom_right_ch] return bayer_image @staticmethod def convert_image_from_BGR(bgr_image: np.ndarray, target_format: str | ImageFormat | None = None) -> tuple[bytes, int, int]: """ Converts a BGR image to the specified target format. Args: bgr_image (np.ndarray): Input image in BGR format. target_format (str | ImageFormat): Target image format. Raises: ValueError: If the target format is unsupported. Returns: A tuple containing the converted image as bytes, height, and width. """ _target_format: ImageFormat = ImageDetails._parse_image_format(target_format) match _target_format: case "BGR8" | ImageFormat.BGR8: res_image = bgr_image case "RGB8" | ImageFormat.RGB8: res_image = cv2.cvtColor(bgr_image, cv2.COLOR_BGR2RGB) case "Mono8" | ImageFormat.Mono8: res_image = cv2.cvtColor(bgr_image, cv2.COLOR_BGR2GRAY) case "BayerRG8" | ImageFormat.BayerRG8: res_image = ImageDetails._bgr_to_bayer(bgr_image, "BayerRG8") case "BayerGR8" | ImageFormat.BayerGR8: res_image = ImageDetails._bgr_to_bayer(bgr_image, "BayerGR8") case "BayerBG8" | ImageFormat.BayerBG8: res_image = ImageDetails._bgr_to_bayer(bgr_image, "BayerBG8") case "BayerGB8" | ImageFormat.BayerGB8: res_image = ImageDetails._bgr_to_bayer(bgr_image, "BayerGB8") case "YUV422Packed" | ImageFormat.YUV422Packed: # COLOR_BGR2YUV_Y422 is the 4:2:2 sampling format of YUV, with coefficients correspond to the BT.601 standard. # After ravel() it becomes a packed format, where the order of the channels is UYVY. res_image = cv2.cvtColor(bgr_image, cv2.COLOR_BGR2YUV_Y422) case "YUV422_YUYV_Packed" | ImageFormat.YUV422_YUYV_Packed: # Same as the above but with YUYV order. res_image = cv2.cvtColor(bgr_image, cv2.COLOR_BGR2YUV_YUYV) case _: raise ValueError(f"Unsupported target format: {target_format}") height, width = res_image.shape[:2] res_image = res_image.ravel().tobytes() return res_image, height, width @staticmethod def from_dict(data: dict): """ Creates an ImageDetails object from the details of a given ImageSet payload. Args: data (dict): Dictionary containing image details Raises: ValueError: If there is an error parsing the image details Returns: ImageDetails: Parsed ImageDetails object """ try: details = ImageDetails( id = data.get("id", f"simaticai-image-{datetime.now().timestamp()}"), seq = data.get("seq", 0), timestamp = parse_aiis_timestamp(data.get("timestamp")), format = ImageFormat[data.get("format", "RGB8")], width = data.get("width", 0), height = data.get("height", 0), metadata = data.get("metadata", {}), image = data.get("image", b"") ) except Exception as e: raise ValueError(f"Error parsing image details: {repr(e)}") return details def to_dict(self) -> dict: """ Converts the ImageDetails object to a dictionary. Returns: dict: Dictionary representation of the ImageDetails object. """ return { "id": self.id, "seq": self.seq, "timestamp": format_aiis_timestamp(self.timestamp), "format": self.format.value, "width": self.width, "height": self.height, "metadata": self.metadata, "image": self.image } def update_image(self, image: np.ndarray, image_format: ImageFormat): """ Updates the image data and format of the ImageDetails object. Args: image (np.ndarray): New image data as a NumPy array. image_format (ImageFormat): New image format. """ self.format = image_format self.height = image.shape[0] self.width = image.shape[1] self.image = image.ravel().tobytes() def get_image_array(self) -> np.ndarray: """Returns the image as a NumPy array based on its format. Raises: ValueError: If the image format is unsupported. Returns: np.ndarray: Image as a NumPy array. """ image = self.image image = np.frombuffer(image, dtype=np.uint8) match self.format: case ImageFormat.RGB8 | ImageFormat.BGR8: image = image.reshape(self.height, self.width, 3) case ImageFormat.Mono8 | ImageFormat.BayerRG8 | ImageFormat.BayerGR8 | ImageFormat.BayerBG8 | ImageFormat.BayerGB8: image = image.reshape(self.height, self.width) case ImageFormat.YUV422Packed | ImageFormat.YUV422_YUYV_Packed: image = image.reshape(self.height, self.width, 2) case _: raise ValueError(f"Unsupported image format: {self.format}") return image def get_image_rgb(self) -> np.ndarray: """Returns the image converted to RGB format as a NumPy array. Raises: ValueError: If the image format is unsupported. Returns: np.ndarray: Image in RGB format as a NumPy array. """ image = self.get_image_array() match self.format: case ImageFormat.Mono8: return cv2.cvtColor(image, cv2.COLOR_GRAY2RGB) case ImageFormat.RGB8: return image case ImageFormat.BGR8: return cv2.cvtColor(image, cv2.COLOR_BGR2RGB) case ImageFormat.BayerRG8: return cv2.cvtColor(image, cv2.COLOR_BayerRGGB2RGB) case ImageFormat.BayerGR8: return cv2.cvtColor(image, cv2.COLOR_BayerGRBG2RGB) case ImageFormat.BayerBG8: return cv2.cvtColor(image, cv2.COLOR_BayerBGGR2RGB) case ImageFormat.BayerGB8: return cv2.cvtColor(image, cv2.COLOR_BayerGBRG2RGB) case ImageFormat.YUV422Packed: return cv2.cvtColor(image, cv2.COLOR_YUV2RGB_Y422) case ImageFormat.YUV422_YUYV_Packed: return cv2.cvtColor(image, cv2.COLOR_YUV2RGB_YUYV) case _: raise ValueError(f"Unsupported image format: {self.format}") ``` ### `from_image(image_path, format=None)` Creates an ImageDetails object from an image file. Args: image_path (Path | str): Path to the image file. format (ImageFormat | str | None): Desired image format (default: RGB8). Raises: ValueError: If the image file is not found or cannot be read. Returns: ImageDetails: Created ImageDetails object. Source code in `docs/industrial-ai-suite/sdk/simaticai/payloads/imageset.py` ```python @staticmethod def from_image(image_path: Path | str, format: ImageFormat | str | None = None) -> 'ImageDetails': """Creates an ImageDetails object from an image file. Args: image_path (Path | str): Path to the image file. format (ImageFormat | str | None): Desired image format (default: RGB8). Raises: ValueError: If the image file is not found or cannot be read. Returns: ImageDetails: Created ImageDetails object. """ format = ImageDetails._parse_image_format(format) image_path = Path(image_path) image = cv2.imread(str(image_path)) if image is None: raise ValueError(f"Image file {image_path} not found") res_image, height, width = ImageDetails.convert_image_from_BGR(image, format) return ImageDetails( id=image_path.name, timestamp=datetime.now(timezone.utc), width=width, height=height, format=format, image=res_image ) ``` ### `convert_image_from_BGR(bgr_image, target_format=None)` Converts a BGR image to the specified target format. Args: bgr_image (np.ndarray): Input image in BGR format. target_format (str | ImageFormat): Target image format. Raises: ValueError: If the target format is unsupported. Returns: A tuple containing the converted image as bytes, height, and width. Source code in `docs/industrial-ai-suite/sdk/simaticai/payloads/imageset.py` ```python @staticmethod def convert_image_from_BGR(bgr_image: np.ndarray, target_format: str | ImageFormat | None = None) -> tuple[bytes, int, int]: """ Converts a BGR image to the specified target format. Args: bgr_image (np.ndarray): Input image in BGR format. target_format (str | ImageFormat): Target image format. Raises: ValueError: If the target format is unsupported. Returns: A tuple containing the converted image as bytes, height, and width. """ _target_format: ImageFormat = ImageDetails._parse_image_format(target_format) match _target_format: case "BGR8" | ImageFormat.BGR8: res_image = bgr_image case "RGB8" | ImageFormat.RGB8: res_image = cv2.cvtColor(bgr_image, cv2.COLOR_BGR2RGB) case "Mono8" | ImageFormat.Mono8: res_image = cv2.cvtColor(bgr_image, cv2.COLOR_BGR2GRAY) case "BayerRG8" | ImageFormat.BayerRG8: res_image = ImageDetails._bgr_to_bayer(bgr_image, "BayerRG8") case "BayerGR8" | ImageFormat.BayerGR8: res_image = ImageDetails._bgr_to_bayer(bgr_image, "BayerGR8") case "BayerBG8" | ImageFormat.BayerBG8: res_image = ImageDetails._bgr_to_bayer(bgr_image, "BayerBG8") case "BayerGB8" | ImageFormat.BayerGB8: res_image = ImageDetails._bgr_to_bayer(bgr_image, "BayerGB8") case "YUV422Packed" | ImageFormat.YUV422Packed: # COLOR_BGR2YUV_Y422 is the 4:2:2 sampling format of YUV, with coefficients correspond to the BT.601 standard. # After ravel() it becomes a packed format, where the order of the channels is UYVY. res_image = cv2.cvtColor(bgr_image, cv2.COLOR_BGR2YUV_Y422) case "YUV422_YUYV_Packed" | ImageFormat.YUV422_YUYV_Packed: # Same as the above but with YUYV order. res_image = cv2.cvtColor(bgr_image, cv2.COLOR_BGR2YUV_YUYV) case _: raise ValueError(f"Unsupported target format: {target_format}") height, width = res_image.shape[:2] res_image = res_image.ravel().tobytes() return res_image, height, width ``` ### `from_dict(data)` Creates an ImageDetails object from the details of a given ImageSet payload. Parameters: | Name | Type | Description | Default | | ------ | ------ | ----------------------------------- | ---------- | | `data` | `dict` | Dictionary containing image details | *required* | Source code in `docs/industrial-ai-suite/sdk/simaticai/payloads/imageset.py` ```python @staticmethod def from_dict(data: dict): """ Creates an ImageDetails object from the details of a given ImageSet payload. Args: data (dict): Dictionary containing image details Raises: ValueError: If there is an error parsing the image details Returns: ImageDetails: Parsed ImageDetails object """ try: details = ImageDetails( id = data.get("id", f"simaticai-image-{datetime.now().timestamp()}"), seq = data.get("seq", 0), timestamp = parse_aiis_timestamp(data.get("timestamp")), format = ImageFormat[data.get("format", "RGB8")], width = data.get("width", 0), height = data.get("height", 0), metadata = data.get("metadata", {}), image = data.get("image", b"") ) except Exception as e: raise ValueError(f"Error parsing image details: {repr(e)}") return details ``` ### `to_dict()` Converts the ImageDetails object to a dictionary. Returns: | Name | Type | Description | | ------ | ------ | ----------------------------------------------------- | | `dict` | `dict` | Dictionary representation of the ImageDetails object. | Source code in `docs/industrial-ai-suite/sdk/simaticai/payloads/imageset.py` ```python def to_dict(self) -> dict: """ Converts the ImageDetails object to a dictionary. Returns: dict: Dictionary representation of the ImageDetails object. """ return { "id": self.id, "seq": self.seq, "timestamp": format_aiis_timestamp(self.timestamp), "format": self.format.value, "width": self.width, "height": self.height, "metadata": self.metadata, "image": self.image } ``` ### `update_image(image, image_format)` Updates the image data and format of the ImageDetails object. Args: image (np.ndarray): New image data as a NumPy array. image_format (ImageFormat): New image format. Source code in `docs/industrial-ai-suite/sdk/simaticai/payloads/imageset.py` ```python def update_image(self, image: np.ndarray, image_format: ImageFormat): """ Updates the image data and format of the ImageDetails object. Args: image (np.ndarray): New image data as a NumPy array. image_format (ImageFormat): New image format. """ self.format = image_format self.height = image.shape[0] self.width = image.shape[1] self.image = image.ravel().tobytes() ``` ### `get_image_array()` Returns the image as a NumPy array based on its format. Raises: ValueError: If the image format is unsupported. Returns: np.ndarray: Image as a NumPy array. Source code in `docs/industrial-ai-suite/sdk/simaticai/payloads/imageset.py` ```python def get_image_array(self) -> np.ndarray: """Returns the image as a NumPy array based on its format. Raises: ValueError: If the image format is unsupported. Returns: np.ndarray: Image as a NumPy array. """ image = self.image image = np.frombuffer(image, dtype=np.uint8) match self.format: case ImageFormat.RGB8 | ImageFormat.BGR8: image = image.reshape(self.height, self.width, 3) case ImageFormat.Mono8 | ImageFormat.BayerRG8 | ImageFormat.BayerGR8 | ImageFormat.BayerBG8 | ImageFormat.BayerGB8: image = image.reshape(self.height, self.width) case ImageFormat.YUV422Packed | ImageFormat.YUV422_YUYV_Packed: image = image.reshape(self.height, self.width, 2) case _: raise ValueError(f"Unsupported image format: {self.format}") return image ``` ### `get_image_rgb()` Returns the image converted to RGB format as a NumPy array. Raises: ValueError: If the image format is unsupported. Returns: np.ndarray: Image in RGB format as a NumPy array. Source code in `docs/industrial-ai-suite/sdk/simaticai/payloads/imageset.py` ```python def get_image_rgb(self) -> np.ndarray: """Returns the image converted to RGB format as a NumPy array. Raises: ValueError: If the image format is unsupported. Returns: np.ndarray: Image in RGB format as a NumPy array. """ image = self.get_image_array() match self.format: case ImageFormat.Mono8: return cv2.cvtColor(image, cv2.COLOR_GRAY2RGB) case ImageFormat.RGB8: return image case ImageFormat.BGR8: return cv2.cvtColor(image, cv2.COLOR_BGR2RGB) case ImageFormat.BayerRG8: return cv2.cvtColor(image, cv2.COLOR_BayerRGGB2RGB) case ImageFormat.BayerGR8: return cv2.cvtColor(image, cv2.COLOR_BayerGRBG2RGB) case ImageFormat.BayerBG8: return cv2.cvtColor(image, cv2.COLOR_BayerBGGR2RGB) case ImageFormat.BayerGB8: return cv2.cvtColor(image, cv2.COLOR_BayerGBRG2RGB) case ImageFormat.YUV422Packed: return cv2.cvtColor(image, cv2.COLOR_YUV2RGB_Y422) case ImageFormat.YUV422_YUYV_Packed: return cv2.cvtColor(image, cv2.COLOR_YUV2RGB_YUYV) case _: raise ValueError(f"Unsupported image format: {self.format}") ``` ## `ImageSet` Data structure representing an ImageSet payload containing multiple images. Fields: version (str): Version of the ImageSet payload. cameraid (str): Unique identifier for the camera. timestamp (datetime): Timestamp when the ImageSet was created. detail (List[ImageDetails]): List of ImageDetails objects representing the images in the set. count (int): Number of images in the ImageSet. Source code in `docs/industrial-ai-suite/sdk/simaticai/payloads/imageset.py` ```python @dataclass class ImageSet: """Data structure representing an ImageSet payload containing multiple images. Fields: version (str): Version of the ImageSet payload. cameraid (str): Unique identifier for the camera. timestamp (datetime): Timestamp when the ImageSet was created. detail (List[ImageDetails]): List of ImageDetails objects representing the images in the set. count (int): Number of images in the ImageSet. """ version: str = field(default="1.0") cameraid: str = field(default_factory=lambda: str(uuid.uuid4())) timestamp: datetime = field(default_factory=lambda: datetime.now(timezone.utc)) detail: List[ImageDetails] = field(default_factory=list) count: int = field(default=0) def __post_init__(self): self.count = len(self.detail) def add_image(self, image: ImageDetails): """Adds an ImageDetails object to the ImageSet. Args: image (ImageDetails): ImageDetails object to add. """ self.detail.append(image) self.count += 1 def get_image_array(self, index: int = 0) -> np.ndarray: """Returns the image at the specified index as a NumPy array. Args: index (int): Index of the image to retrieve (default: 0). Raises: IndexError: If the index is out of range. Returns: np.ndarray: Image as a NumPy array. """ if index < 0 or index >= len(self.detail): raise IndexError("Index out of range") image = self.detail[index].get_image_array() return image def get_image_rgb(self, index: int = 0) -> np.ndarray: """Returns the image at the specified index converted to RGB format as a NumPy array. Args: index (int): Index of the image to retrieve (default: 0). Raises: IndexError: If the index is out of range. Returns: np.ndarray: Image in RGB format as a NumPy array. """ if index < 0 or index >= len(self.detail): raise IndexError("Index out of range") return self.detail[index].get_image_rgb() def to_dict(self): """Converts the ImageSet object to a dictionary. Returns: dict: Dictionary representation of the ImageSet object. """ return { "version": self.version, "count": self.count, "cameraid": str(self.cameraid), "timestamp": format_aiis_timestamp(self.timestamp), "detail": [image.to_dict() for image in self.detail] } @staticmethod def from_dict(data: dict): """Creates an ImageSet object from a dictionary payload. Args: data (dict): Dictionary containing ImageSet data. Raises: ValueError: If there is an error parsing the ImageSet data. Returns: ImageSet: Parsed ImageSet object. """ detail = data.get("detail", []) return ImageSet( version=data.get("version", "1.0"), count=data.get("count", len(detail)), cameraid=data.get("cameraid", str(uuid.uuid4())), timestamp=parse_aiis_timestamp(data.get("timestamp")), detail=[ImageDetails.from_dict(image) for image in detail] ) ``` ### `add_image(image)` Adds an ImageDetails object to the ImageSet. Args: image (ImageDetails): ImageDetails object to add. Source code in `docs/industrial-ai-suite/sdk/simaticai/payloads/imageset.py` ```python def add_image(self, image: ImageDetails): """Adds an ImageDetails object to the ImageSet. Args: image (ImageDetails): ImageDetails object to add. """ self.detail.append(image) self.count += 1 ``` ### `get_image_array(index=0)` Returns the image at the specified index as a NumPy array. Args: index (int): Index of the image to retrieve (default: 0). Raises: IndexError: If the index is out of range. Returns: np.ndarray: Image as a NumPy array. Source code in `docs/industrial-ai-suite/sdk/simaticai/payloads/imageset.py` ```python def get_image_array(self, index: int = 0) -> np.ndarray: """Returns the image at the specified index as a NumPy array. Args: index (int): Index of the image to retrieve (default: 0). Raises: IndexError: If the index is out of range. Returns: np.ndarray: Image as a NumPy array. """ if index < 0 or index >= len(self.detail): raise IndexError("Index out of range") image = self.detail[index].get_image_array() return image ``` ### `get_image_rgb(index=0)` Returns the image at the specified index converted to RGB format as a NumPy array. Args: index (int): Index of the image to retrieve (default: 0). Raises: IndexError: If the index is out of range. Returns: np.ndarray: Image in RGB format as a NumPy array. Source code in `docs/industrial-ai-suite/sdk/simaticai/payloads/imageset.py` ```python def get_image_rgb(self, index: int = 0) -> np.ndarray: """Returns the image at the specified index converted to RGB format as a NumPy array. Args: index (int): Index of the image to retrieve (default: 0). Raises: IndexError: If the index is out of range. Returns: np.ndarray: Image in RGB format as a NumPy array. """ if index < 0 or index >= len(self.detail): raise IndexError("Index out of range") return self.detail[index].get_image_rgb() ``` ### `to_dict()` Converts the ImageSet object to a dictionary. Returns: dict: Dictionary representation of the ImageSet object. Source code in `docs/industrial-ai-suite/sdk/simaticai/payloads/imageset.py` ```python def to_dict(self): """Converts the ImageSet object to a dictionary. Returns: dict: Dictionary representation of the ImageSet object. """ return { "version": self.version, "count": self.count, "cameraid": str(self.cameraid), "timestamp": format_aiis_timestamp(self.timestamp), "detail": [image.to_dict() for image in self.detail] } ``` ### `from_dict(data)` Creates an ImageSet object from a dictionary payload. Args: data (dict): Dictionary containing ImageSet data. Raises: ValueError: If there is an error parsing the ImageSet data. Returns: ImageSet: Parsed ImageSet object. Source code in `docs/industrial-ai-suite/sdk/simaticai/payloads/imageset.py` ```python @staticmethod def from_dict(data: dict): """Creates an ImageSet object from a dictionary payload. Args: data (dict): Dictionary containing ImageSet data. Raises: ValueError: If there is an error parsing the ImageSet data. Returns: ImageSet: Parsed ImageSet object. """ detail = data.get("detail", []) return ImageSet( version=data.get("version", "1.0"), count=data.get("count", len(detail)), cameraid=data.get("cameraid", str(uuid.uuid4())), timestamp=parse_aiis_timestamp(data.get("timestamp")), detail=[ImageDetails.from_dict(image) for image in detail] ) ``` ## `parse_aiis_timestamp(timestamp_str)` Parses a timestamp string from AI Inference Server format to a datetime object. Args: timestamp_str (str | None): Timestamp string in AI Inference Server format. Returns: datetime: Parsed datetime object. Source code in `docs/industrial-ai-suite/sdk/simaticai/payloads/imageset.py` ```python def parse_aiis_timestamp(timestamp_str: str | None) -> datetime: """Parses a timestamp string from AI Inference Server format to a datetime object. Args: timestamp_str (str | None): Timestamp string in AI Inference Server format. Returns: datetime: Parsed datetime object. """ if timestamp_str is None or "" == timestamp_str.strip(): return datetime.now(timezone.utc) timestamp_str = re.sub(r'[zZ]$', '+00:00', timestamp_str) return datetime.strptime(timestamp_str, DATE_TIME_FORMAT) ``` ## `format_aiis_timestamp(timestamp)` Formats a datetime object to a timestamp string in AI Inference Server format. Args: timestamp (datetime | None): Datetime object to format. Returns: str: Formatted timestamp string. Source code in `docs/industrial-ai-suite/sdk/simaticai/payloads/imageset.py` ```python def format_aiis_timestamp(timestamp: datetime | None) -> str: """Formats a datetime object to a timestamp string in AI Inference Server format. Args: timestamp (datetime | None): Datetime object to format. Returns: str: Formatted timestamp string. """ if timestamp is None: timestamp = datetime.now(timezone.utc) return re.sub(r'\+.*$','', timestamp.isoformat(timespec='milliseconds')) + 'Z' ``` Contains classes for defining pipeline variables and parameters, including their connections and mappings. PipelineVariable links a component's variable to a connection, while PipelineParameter defines parameters that can be set runtime. PipelineParameter can be topic-based, allowing dynamic updates via messaging topics. SecretPipelineParameter is a specialized parameter type for handling sensitive information. ## `MappableMixin` Mixin class providing mapping functionality for preconfiguration of Pipeline Variables and Parameters. Attributes: | Name | Type | Description | | ------------ | ---------------------- | -------------------------------------------- | | `connection` | `Optional[Connection]` | The connection associated with the variable. | | `name` | `Optional[str]` | The tag or camera name to map to. | | `mapping` | `Optional[str]` | The mapping to map to. | Methods: | Name | Description | | ---------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `add_mapping` | Optional[str], cameraName: Optional[str], mapping: Optional[str]) -> Self: Adds mapping information to the variable. | | `add_connection` | Connection) -> Self: Adds a connection to the variable. | | `add_mapping_with_connector` | Connection, tagName: Optional[str], cameraName: Optional[str], mapping: Optional[str]) -> Self: Adds both mapping information and a connection to the variable. | | `_to_config_mapping` | Returns the mapping configuration as a dictionary. | Source code in `docs/industrial-ai-suite/sdk/simaticai/payloads/pipeline_variable.py` ```python class MappableMixin: """ Mixin class providing mapping functionality for preconfiguration of Pipeline Variables and Parameters. Attributes: connection (Optional[Connection]): The connection associated with the variable. name (Optional[str]): The tag or camera name to map to. mapping (Optional[str]): The mapping to map to. Methods: add_mapping(tagName: Optional[str], cameraName: Optional[str], mapping: Optional[str]) -> Self: Adds mapping information to the variable. add_connection(connection: Connection) -> Self: Adds a connection to the variable. add_mapping_with_connector(connection: Connection, tagName: Optional[str], cameraName: Optional[str], mapping: Optional[str]) -> Self: Adds both mapping information and a connection to the variable. _to_config_mapping() -> dict: Returns the mapping configuration as a dictionary. """ def __setattr__(self, name, value): """Handle setting of mapping-related attributes.""" match name: case "connection": if value is not None and not isinstance(value, Connection): raise ValueError("ERROR! 'connection' must be an instance of Connection or None.") case "tagName" | "cameraName" | "mapping": if hasattr(self, 'topicBased') and value is not None: self.topicBased = True if name == "cameraName" and value is not None: self.tagName = value case _: # Other attributes are handled by the dataclass pass super().__setattr__(name, value) def add_mapping(self, *, tagName: Optional[str] = None, cameraName: Optional[str] = None, mapping: Optional[str] = None): """ Adds mapping information to the variable. Property 'tagName' is supposed to be used with message queues, while 'cameraName' with Vision Connector Application. In case you define both, only 'tagName' will be used and preconfigured. Args: tagName (Optional[str]): The tag name to map to. cameraName (Optional[str]): The camera name to map to. mapping (Optional[str]): The topic or camera id to map to. """ self.tagName = tagName or cameraName self.mapping = mapping return self def add_connection(self, connection: Connection): """ Adds a connection to the variable. Args: connection (Connection): The connection to add. """ self.connection = connection return self def add_mapping_with_connection(self, connection: Connection, *, tagName: Optional[str] = None, cameraName: Optional[str] = None, mapping: Optional[str] = None): """ Adds both mapping information and a connection to the variable. Property 'tagName' is supposed to be used with message queues, while 'cameraName' with Vision Connector Application. In case you define both, only 'tagName' will be used and preconfigured. Args: connection (Connection): The connection to add. tagName (Optional[str]): The tag name to map to. cameraName (Optional[str]): The camera name to map to. mapping (Optional[str]): The topic or camera id to map to. """ self.add_connection(connection) self.add_mapping(tagName=tagName, cameraName=cameraName, mapping=mapping) return self def __mappable_dict__(self) -> dict: """Returns the mapping configuration as a dictionary.""" json_dict = {} if self.connection is not None: for k, v in self.connection.__dict__().items(): json_dict[k] = v if self.tagName is not None: json_dict['tagName'] = self.tagName if self.mapping is not None: json_dict['topic'] = self.mapping return json_dict ``` ### `__setattr__(name, value)` Handle setting of mapping-related attributes. Source code in `docs/industrial-ai-suite/sdk/simaticai/payloads/pipeline_variable.py` ```python def __setattr__(self, name, value): """Handle setting of mapping-related attributes.""" match name: case "connection": if value is not None and not isinstance(value, Connection): raise ValueError("ERROR! 'connection' must be an instance of Connection or None.") case "tagName" | "cameraName" | "mapping": if hasattr(self, 'topicBased') and value is not None: self.topicBased = True if name == "cameraName" and value is not None: self.tagName = value case _: # Other attributes are handled by the dataclass pass super().__setattr__(name, value) ``` ### `add_mapping(*, tagName=None, cameraName=None, mapping=None)` Adds mapping information to the variable. Property 'tagName' is supposed to be used with message queues, while 'cameraName' with Vision Connector Application. In case you define both, only 'tagName' will be used and preconfigured. Parameters: | Name | Type | Description | Default | | ------------ | --------------- | --------------------------------- | ------- | | `tagName` | `Optional[str]` | The tag name to map to. | `None` | | `cameraName` | `Optional[str]` | The camera name to map to. | `None` | | `mapping` | `Optional[str]` | The topic or camera id to map to. | `None` | Source code in `docs/industrial-ai-suite/sdk/simaticai/payloads/pipeline_variable.py` ```python def add_mapping(self, *, tagName: Optional[str] = None, cameraName: Optional[str] = None, mapping: Optional[str] = None): """ Adds mapping information to the variable. Property 'tagName' is supposed to be used with message queues, while 'cameraName' with Vision Connector Application. In case you define both, only 'tagName' will be used and preconfigured. Args: tagName (Optional[str]): The tag name to map to. cameraName (Optional[str]): The camera name to map to. mapping (Optional[str]): The topic or camera id to map to. """ self.tagName = tagName or cameraName self.mapping = mapping return self ``` ### `add_connection(connection)` Adds a connection to the variable. Args: connection (Connection): The connection to add. Source code in `docs/industrial-ai-suite/sdk/simaticai/payloads/pipeline_variable.py` ```python def add_connection(self, connection: Connection): """ Adds a connection to the variable. Args: connection (Connection): The connection to add. """ self.connection = connection return self ``` ### `add_mapping_with_connection(connection, *, tagName=None, cameraName=None, mapping=None)` Adds both mapping information and a connection to the variable. Property 'tagName' is supposed to be used with message queues, while 'cameraName' with Vision Connector Application. In case you define both, only 'tagName' will be used and preconfigured. Parameters: | Name | Type | Description | Default | | ------------ | --------------- | --------------------------------- | ---------- | | `connection` | `Connection` | The connection to add. | *required* | | `tagName` | `Optional[str]` | The tag name to map to. | `None` | | `cameraName` | `Optional[str]` | The camera name to map to. | `None` | | `mapping` | `Optional[str]` | The topic or camera id to map to. | `None` | Source code in `docs/industrial-ai-suite/sdk/simaticai/payloads/pipeline_variable.py` ```python def add_mapping_with_connection(self, connection: Connection, *, tagName: Optional[str] = None, cameraName: Optional[str] = None, mapping: Optional[str] = None): """ Adds both mapping information and a connection to the variable. Property 'tagName' is supposed to be used with message queues, while 'cameraName' with Vision Connector Application. In case you define both, only 'tagName' will be used and preconfigured. Args: connection (Connection): The connection to add. tagName (Optional[str]): The tag name to map to. cameraName (Optional[str]): The camera name to map to. mapping (Optional[str]): The topic or camera id to map to. """ self.add_connection(connection) self.add_mapping(tagName=tagName, cameraName=cameraName, mapping=mapping) return self ``` ### `__mappable_dict__()` Returns the mapping configuration as a dictionary. Source code in `docs/industrial-ai-suite/sdk/simaticai/payloads/pipeline_variable.py` ```python def __mappable_dict__(self) -> dict: """Returns the mapping configuration as a dictionary.""" json_dict = {} if self.connection is not None: for k, v in self.connection.__dict__().items(): json_dict[k] = v if self.tagName is not None: json_dict['tagName'] = self.tagName if self.mapping is not None: json_dict['topic'] = self.mapping return json_dict ``` ## `PipelineVariable` Bases: `MappableMixin` Represents a variable in a pipeline, linking a component's variable to a connection and optional mapping. For IE Vision connections, cameraName should be defined, otherwise tagName is preferred. Attributes: | Name | Type | Description | | --------------- | ---------------------- | -------------------------------------------------------------- | | `componentName` | `str` | The name of the component the variable belongs to. | | `variableName` | `str` | The name of the variable. | | `connection` | `Optional[Connection]` | The connection associated with the variable. | | `tagName` | `Optional[str]` | The tag name to map to. | | `cameraName` | `Optional[str]` | The camera name to map to. Used only via IE_Vision connection. | | `mapping` | `Optional[str]` | The topic or camera id to map to. | Source code in `docs/industrial-ai-suite/sdk/simaticai/payloads/pipeline_variable.py` ```python @dataclass class PipelineVariable(MappableMixin): """ Represents a variable in a pipeline, linking a component's variable to a connection and optional mapping. For IE Vision connections, cameraName should be defined, otherwise tagName is preferred. Attributes: componentName (str): The name of the component the variable belongs to. variableName (str): The name of the variable. connection (Optional[Connection]): The connection associated with the variable. tagName (Optional[str]): The tag name to map to. cameraName (Optional[str]): The camera name to map to. Used only via IE_Vision connection. mapping (Optional[str]): The topic or camera id to map to. """ componentName: str variableName: str # Mixin fields as dataclass fields connection: Optional[Connection] = field(default=None) tagName: Optional[str] = field(default=None) cameraName: Optional[str] = field(default=None) mapping: Optional[str] = field(default=None) def check_connection_type_compatibility(self, variable_type: str) -> tuple[list, list]: if self.connection is None: return [], [] supported_types = SUPPORTED_TYPES_BY_CONNECTION.get(self.connection.cptype, []) flags = type_validation_flags(variable_type, supported_types) if self.connection.cptype == ConnectionTypeAndPayloadFormat.Realtime_Backbone: warning_flags = [] error_flags = [f.value for f in flags] else: warning_flags = [f.value for f in flags if f != TypeValidationFlags.ARRAY_WITH_MISSING_SIZE] error_flags = [] return warning_flags, error_flags def __getitem__(self, item): return getattr(self, item) def __setattr__(self, name, value): """ Setter for all attributes """ match name: case "componentName": if re.match("^[a-zA-Z0-9_-]+$", value) is None: raise ValueError(f"ERROR! Invalid name '{value}'. Only alphanumeric characters and underscores are allowed.") case "variableName": if re.match("^[a-zA-Z0-9_-]+$", value) is None: raise ValueError(f"ERROR! Invalid name '{value}'. Only alphanumeric characters and underscores are allowed.") if value in reserved_names: raise ValueError(f"ERROR! '{value}' is a reserved name and cannot be used as variable name.") case _: # Let the mixin handle mapping attributes pass super().__setattr__(name, value) def __dict__(self, component_io_list: list) -> dict: json_dict = { "name": self.variableName, "type": component_io_list[self.variableName]['type'] } json_dict.update(self.__mappable_dict__()) return json_dict ``` ### `__setattr__(name, value)` Setter for all attributes Source code in `docs/industrial-ai-suite/sdk/simaticai/payloads/pipeline_variable.py` ```python def __setattr__(self, name, value): """ Setter for all attributes """ match name: case "componentName": if re.match("^[a-zA-Z0-9_-]+$", value) is None: raise ValueError(f"ERROR! Invalid name '{value}'. Only alphanumeric characters and underscores are allowed.") case "variableName": if re.match("^[a-zA-Z0-9_-]+$", value) is None: raise ValueError(f"ERROR! Invalid name '{value}'. Only alphanumeric characters and underscores are allowed.") if value in reserved_names: raise ValueError(f"ERROR! '{value}' is a reserved name and cannot be used as variable name.") case _: # Let the mixin handle mapping attributes pass super().__setattr__(name, value) ``` ## `PipelineParameter` Bases: `MappableMixin` Source code in `docs/industrial-ai-suite/sdk/simaticai/payloads/pipeline_variable.py` ```python @dataclass class PipelineParameter(MappableMixin): name: str defaultValue: Optional[str] = None dtype: Optional[str] = None # 'String', 'Integer', 'Float', 'Boolean' description: Optional[str] = None topicBased: Optional[bool] = False valueTopic: Optional[str] = None # Mixin fields as dataclass fields connection: Optional[Connection] = field(default=None) tagName: Optional[str] = field(default=None) mapping: Optional[str] = field(default=None) def __post_init__(self): # Validate if the topicBased flag is False, then valueTopic, connection, tag, and topic should not be set if not self.topicBased and (self.valueTopic is not None or self.connection is not None or self.tagName is not None or self.mapping is not None): raise ValueError("ERROR! 'valueTopic', 'connection', 'tag', and 'topic' must be None when 'topicBased' is False.") if self.dtype != _get_is_type(type(self.defaultValue)): raise ValueError(f"'The given value type does not match the type of defaultValue: '{self.defaultValue}'") def __setattr__(self, name, value): """ Setter for all attributes """ match name: case "name": _check_name_validity(value) case "dtype": if value is None: return elif value not in ['String', 'Integer', 'Double', 'Boolean']: raise ValueError("ERROR! 'dtype' must be one of 'String', 'Integer', 'Double', 'Boolean', or None.") case "defaultValue": dtype = _get_is_type(type(value)) if dtype is None: raise ValueError("ERROR! 'dtype' must be one of 'String', 'Integer', 'Double', 'Boolean', or None.") self.dtype = dtype case "description": _check_description_validity(value) case "topicBased": if not isinstance(value, bool): raise ValueError("Type of the given `topic_based` parameter is not `bool`") case "valueTopic": if not (value is None or value == ""): self.topicBased = True case _: # Let the mixin handle mapping attributes pass super().__setattr__(name, value) def has_valid_connection(self) -> set: if self.connection is None: return True supported_types = SUPPORTED_TYPES_BY_CONNECTION.get(self.connection.cptype, []) return type_validation_flags(self.dtype, supported_types) == set() def __param_dict__(self) -> dict: json_dict = { 'name': self.name, 'defaultValue': self.defaultValue, 'dtype': self.dtype } json_dict.update(self.__mappable_dict__()) json_dict['type'] = json_dict.pop('dtype', None) if self.topicBased: json_dict['topicBased'] = True json_dict['valueTopic'] = json_dict.pop('topic', None) return json_dict ``` ### `__setattr__(name, value)` Setter for all attributes Source code in `docs/industrial-ai-suite/sdk/simaticai/payloads/pipeline_variable.py` ```python def __setattr__(self, name, value): """ Setter for all attributes """ match name: case "name": _check_name_validity(value) case "dtype": if value is None: return elif value not in ['String', 'Integer', 'Double', 'Boolean']: raise ValueError("ERROR! 'dtype' must be one of 'String', 'Integer', 'Double', 'Boolean', or None.") case "defaultValue": dtype = _get_is_type(type(value)) if dtype is None: raise ValueError("ERROR! 'dtype' must be one of 'String', 'Integer', 'Double', 'Boolean', or None.") self.dtype = dtype case "description": _check_description_validity(value) case "topicBased": if not isinstance(value, bool): raise ValueError("Type of the given `topic_based` parameter is not `bool`") case "valueTopic": if not (value is None or value == ""): self.topicBased = True case _: # Let the mixin handle mapping attributes pass super().__setattr__(name, value) ``` ## `SecretPipelineParameter` Represents a secret pipeline parameter for handling sensitive information. Cannot be modified after creation. Attributes: | Name | Type | Description | | -------------- | --------------- | ---------------------------------------------------------------------------------------------- | | `name` | `str` | The name of the secret parameter. | | `description` | `Optional[str]` | An optional description of the secret parameter. | | `defaultValue` | `str` | Pipeline parameters must have a default value, which is an empty string for secret parameters. | | `dtype` | `str` | The data type of the secret parameter is always "Secret". | Source code in `docs/industrial-ai-suite/sdk/simaticai/payloads/pipeline_variable.py` ```python @dataclass(frozen=True) class SecretPipelineParameter: """ Represents a secret pipeline parameter for handling sensitive information. Cannot be modified after creation. Attributes: name (str): The name of the secret parameter. description (Optional[str]): An optional description of the secret parameter. defaultValue (str): Pipeline parameters must have a default value, which is an empty string for secret parameters. dtype (str): The data type of the secret parameter is always "Secret". """ name: str description: Optional[str] = None defaultValue: str = field(init=False, default="") dtype: str = field(init=False, default="Secret") def __post_init__(self): _check_name_validity(self.name) _check_description_validity(self.description) def __param_dict__(self) -> dict: json_dict = { 'name': self.name, 'defaultValue': "", 'type': self.dtype } return json_dict ``` # telemetry Telemetry Consent Management Module This module provides functionality for managing user consent for telemetry data collection in the SIMATIC AI SDK. It implements a hierarchical consent system that respects user privacy choices across multiple configuration methods. Consent Decision Hierarchy 1. Session-level overrides (allow_telemetry_once/deny_telemetry_once) 1. Environment variable (SIMATICAI_TELEMETRY) 1. Configuration file (consent.yml in user config directory) Key Features - Persistent consent storage in YAML configuration files - Temporary session-level consent overrides - Environment variable support for CI/CD pipelines - Command-line interface for consent management - Automatic tracking of consent method, timestamp, and SDK version Usage Check if telemetry is allowed: > > > from simaticai.telemetry import is_telemetry_allowed if is_telemetry_allowed(): ... # Send telemetry data ... pass Set consent programmatically (persistent): > > > from simaticai.telemetry import allow_telemetry, deny_telemetry allow_telemetry() # Save consent to config file deny_telemetry() # Save denial to config file Set consent for current session only: > > > from simaticai.telemetry import allow_telemetry_once, deny_telemetry_once allow_telemetry_once() # Temporary, not saved deny_telemetry_once() # Temporary, not saved Via environment variable: $ export SIMATICAI_TELEMETRY=true $ python my_script.py Via command line: $ simaticai telemetry --allow $ simaticai telemetry --deny $ simaticai telemetry --status Configuration File The consent configuration is stored in a platform-specific user configuration directory (e.g., ~/.config/simaticai/consent.yml on Linux) and includes: - User's consent decision (allow_telemetry: true/false) - Timestamp of the decision - SDK version at time of consent - Method used to provide consent - Full consent information text Exception Handling If no consent decision has been made, calling is_telemetry_allowed() will raise a RuntimeError prompting the user to provide consent via the CLI command: $ simaticai telemetry ## `get_consent_source()` Get the source of the telemetry consent decision. Returns: | Name | Type | Description | | ----- | ----- | ---------------------------------------------------------------------------------------------- | | `str` | `str` | The source of consent (e.g., 'python method', 'environment variable', 'command line', 'none'). | Source code in `docs/industrial-ai-suite/sdk/simaticai/telemetry/consent.py` ```python def get_consent_source() -> str: """Get the source of the telemetry consent decision. Returns: str: The source of consent (e.g., 'python method', 'environment variable', 'command line', 'none'). """ return _CONSENT_SOURCE ``` ## `get_user_config_dir_path()` Get the user configuration directory path for the application. Creates the directory if it doesn't exist. Returns: | Name | Type | Description | | ------ | ------ | ----------------------------------------------------------- | | `Path` | `Path` | Path object pointing to the user's configuration directory. | Source code in `docs/industrial-ai-suite/sdk/simaticai/telemetry/consent.py` ```python def get_user_config_dir_path() -> Path: """Get the user configuration directory path for the application. Creates the directory if it doesn't exist. Returns: Path: Path object pointing to the user's configuration directory. """ return platformdirs.user_config_path(appname=_APP_NAME, appauthor=_APP_AUTHOR, roaming=True, ensure_exists=True) ``` ## `is_telemetry_allowed_by_env_var()` Check if telemetry is allowed via environment variable. Checks the SIMATICAI_TELEMETRY environment variable for consent. Returns: | Name | Type | Description | | --------------- | --------------- | ----------------------------------------------------------------------------------------------------------- | | `ConsentStatus` | `ConsentStatus` | ALLOWED if env var is set to an affirmative value, DENIED if set to a negative value, UNDECIDED if not set. | Source code in `docs/industrial-ai-suite/sdk/simaticai/telemetry/consent.py` ```python def is_telemetry_allowed_by_env_var() -> ConsentStatus: """Check if telemetry is allowed via environment variable. Checks the SIMATICAI_TELEMETRY environment variable for consent. Returns: ConsentStatus: ALLOWED if env var is set to an affirmative value, DENIED if set to a negative value, UNDECIDED if not set. """ if _CONSENT_ENV_VAR not in os.environ: return ConsentStatus.UNDECIDED choice = os.getenv(_CONSENT_ENV_VAR, "").lower() in _CONSENT_AFFIRMATIVE _logger.warning(f"Telemetry is {'allowed' if choice else 'denied'} by environment variable.") if choice: return ConsentStatus.ALLOWED return ConsentStatus.DENIED ``` ## `read_telemetry_consent_file()` Read the telemetry consent configuration file. Returns: | Type | Description | | ------------------- | ----------------------------------------------------------------------------------------------------- | | `tuple[dict, Path]` | tuple\[dict, Path\]: A tuple containing the consent data dictionary and the path to the consent file. | Raises: | Type | Description | | ---------------- | ----------------------------------------------------------------- | | `AssertionError` | If the consent file exists but the user's choice is not readable. | Source code in `docs/industrial-ai-suite/sdk/simaticai/telemetry/consent.py` ```python def read_telemetry_consent_file() -> tuple[dict, Path]: """Read the telemetry consent configuration file. Returns: tuple[dict, Path]: A tuple containing the consent data dictionary and the path to the consent file. Raises: AssertionError: If the consent file exists but the user's choice is not readable. """ consent_file = get_user_config_dir_path() / _CONSENT_FILE_NAME data = yaml_helper.read_yaml(consent_file) if CONSENT_KEY not in data: raise AssertionError("Consent file exists, but the user's choice is not readable.") return data, consent_file ``` ## `is_telemetry_allowed_by_config_file()` Check if telemetry is allowed via configuration file. Reads the consent file and returns the user's telemetry choice. Returns: | Name | Type | Description | | ------ | ------ | ---------------------------------------------- | | `bool` | `bool` | True if telemetry is allowed, False otherwise. | Raises: | Type | Description | | -------------- | --------------------------------------------------------------------------------------------------- | | `RuntimeError` | If the consent file cannot be read or doesn't exist, prompting the user to make a consent decision. | Source code in `docs/industrial-ai-suite/sdk/simaticai/telemetry/consent.py` ```python def is_telemetry_allowed_by_config_file() -> bool: """Check if telemetry is allowed via configuration file. Reads the consent file and returns the user's telemetry choice. Returns: bool: True if telemetry is allowed, False otherwise. Raises: RuntimeError: If the consent file cannot be read or doesn't exist, prompting the user to make a consent decision. """ try: data, _ = read_telemetry_consent_file() choice = data.get(CONSENT_KEY, False) if not isinstance(choice, bool): choice = str(choice).lower() in _CONSENT_AFFIRMATIVE _logger.warning(f"Telemetry is {'allowed' if choice else 'denied'} by configuration file.") return choice except BaseException as error: global _CONSENT_SOURCE _CONSENT_SOURCE = "none" raise RuntimeError(f"\n\n{CONSENT_EXCEPTION_TEXT}\n") from error ``` ## `is_telemetry_allowed()` Check if telemetry is allowed based on all possible sources. Checks consent in order of precedence: 1. Override set via allow_telemetry_once/deny_telemetry_once 1. Environment variable (SIMATICAI_TELEMETRY) 1. Configuration file Returns: | Name | Type | Description | | ------ | ------ | ---------------------------------------------- | | `bool` | `bool` | True if telemetry is allowed, False otherwise. | Raises: | Type | Description | | -------------- | ----------------------------------------------------------------- | | `RuntimeError` | If no consent decision has been made and no configuration exists. | Source code in `docs/industrial-ai-suite/sdk/simaticai/telemetry/consent.py` ```python def is_telemetry_allowed() -> bool: """Check if telemetry is allowed based on all possible sources. Checks consent in order of precedence: 1. Override set via allow_telemetry_once/deny_telemetry_once 2. Environment variable (SIMATICAI_TELEMETRY) 3. Configuration file Returns: bool: True if telemetry is allowed, False otherwise. Raises: RuntimeError: If no consent decision has been made and no configuration exists. """ global _CONSENT_SOURCE _CONSENT_SOURCE = "python method" if ConsentStatus.UNDECIDED != _CONSENT_OVERRIDE: return ConsentStatus.ALLOWED == _CONSENT_OVERRIDE _CONSENT_SOURCE = "environment variable" choice_env = is_telemetry_allowed_by_env_var() if ConsentStatus.UNDECIDED != choice_env: return ConsentStatus.ALLOWED == choice_env _CONSENT_SOURCE = "command line" return is_telemetry_allowed_by_config_file() ``` ## `set_telemetry_consent(allow)` Set the telemetry consent decision in the configuration file. Creates or updates the consent file with the user's choice, timestamp, SDK version, and consent method. Parameters: | Name | Type | Description | Default | | ------- | ------ | ------------------------------------------ | ---------- | | `allow` | `bool` | True to allow telemetry, False to deny it. | *required* | Returns: | Name | Type | Description | | ------ | ------ | ----------------------------------------------------- | | `Path` | `Path` | Path to the consent file that was created or updated. | Raises: | Type | Description | | -------------- | -------------------------------------- | | `RuntimeError` | If the consent file cannot be written. | Source code in `docs/industrial-ai-suite/sdk/simaticai/telemetry/consent.py` ```python def set_telemetry_consent(allow: bool) -> Path: """Set the telemetry consent decision in the configuration file. Creates or updates the consent file with the user's choice, timestamp, SDK version, and consent method. Args: allow (bool): True to allow telemetry, False to deny it. Returns: Path: Path to the consent file that was created or updated. Raises: RuntimeError: If the consent file cannot be written. """ consent_file = get_user_config_dir_path() / _CONSENT_FILE_NAME try: with open(consent_file, "w") as f: # writing YAML manually to keep the file format as intended f.write(f"{CONSENT_KEY}: {'true' if allow else 'false'}\n") f.write(f"{CONSENT_KEY_TS}: '{datetime.now().isoformat()}'\n") f.write(f"{CONSENT_KEY_SDK}: '{version('simaticai')}'\n") f.write(f"{CONSENT_KEY_METHOD}: '{_CONSENT_SOURCE}'\n") f.write("information: |\n") for line in CONSENT_INFO.splitlines(): f.write(f" {line}\n") except BaseException as error: raise RuntimeError(f"Failed to save consent in '{consent_file}'.") from error return consent_file ``` ## `allow_telemetry()` Allow telemetry and save the decision to the configuration file. Sets the consent source to 'python method' and saves the consent decision. Returns: | Name | Type | Description | | ------ | ------ | ------------------------------------------ | | `Path` | `Path` | Path to the consent file that was updated. | Source code in `docs/industrial-ai-suite/sdk/simaticai/telemetry/consent.py` ```python def allow_telemetry() -> Path: """Allow telemetry and save the decision to the configuration file. Sets the consent source to 'python method' and saves the consent decision. Returns: Path: Path to the consent file that was updated. """ global _CONSENT_SOURCE _CONSENT_SOURCE = "python method" return set_telemetry_consent(True) ``` ## `deny_telemetry()` Deny telemetry and save the decision to the configuration file. Sets the consent source to 'python method' and saves the consent decision. Returns: | Name | Type | Description | | ------ | ------ | ------------------------------------------ | | `Path` | `Path` | Path to the consent file that was updated. | Source code in `docs/industrial-ai-suite/sdk/simaticai/telemetry/consent.py` ```python def deny_telemetry() -> Path: """Deny telemetry and save the decision to the configuration file. Sets the consent source to 'python method' and saves the consent decision. Returns: Path: Path to the consent file that was updated. """ global _CONSENT_SOURCE _CONSENT_SOURCE = "python method" return set_telemetry_consent(False) ``` ## `allow_telemetry_cli()` Allow telemetry via command line interface and save the decision. Sets the consent source to 'command line' and saves the consent decision. Returns: | Name | Type | Description | | ------ | ------ | ------------------------------------------ | | `Path` | `Path` | Path to the consent file that was updated. | Source code in `docs/industrial-ai-suite/sdk/simaticai/telemetry/consent.py` ```python def allow_telemetry_cli() -> Path: """Allow telemetry via command line interface and save the decision. Sets the consent source to 'command line' and saves the consent decision. Returns: Path: Path to the consent file that was updated. """ global _CONSENT_SOURCE _CONSENT_SOURCE = "command line" return set_telemetry_consent(True) ``` ## `deny_telemetry_cli()` Deny telemetry via command line interface and save the decision. Sets the consent source to 'command line' and saves the consent decision. Returns: | Name | Type | Description | | ------ | ------ | ------------------------------------------ | | `Path` | `Path` | Path to the consent file that was updated. | Source code in `docs/industrial-ai-suite/sdk/simaticai/telemetry/consent.py` ```python def deny_telemetry_cli() -> Path: """Deny telemetry via command line interface and save the decision. Sets the consent source to 'command line' and saves the consent decision. Returns: Path: Path to the consent file that was updated. """ global _CONSENT_SOURCE _CONSENT_SOURCE = "command line" return set_telemetry_consent(False) ``` ## `allow_telemetry_once()` Allow telemetry for the current session only without saving to file. Sets a temporary override that allows telemetry for this session. The decision is not persisted to the configuration file. Source code in `docs/industrial-ai-suite/sdk/simaticai/telemetry/consent.py` ```python def allow_telemetry_once(): """Allow telemetry for the current session only without saving to file. Sets a temporary override that allows telemetry for this session. The decision is not persisted to the configuration file. """ global _CONSENT_OVERRIDE, _CONSENT_SOURCE _CONSENT_SOURCE = "python method" _CONSENT_OVERRIDE = ConsentStatus.ALLOWED _logger.warning("Telemetry is enabled for this session.") ``` ## `deny_telemetry_once()` Deny telemetry for the current session only without saving to file. Sets a temporary override that denies telemetry for this session. The decision is not persisted to the configuration file. Source code in `docs/industrial-ai-suite/sdk/simaticai/telemetry/consent.py` ```python def deny_telemetry_once(): """Deny telemetry for the current session only without saving to file. Sets a temporary override that denies telemetry for this session. The decision is not persisted to the configuration file. """ global _CONSENT_OVERRIDE, _CONSENT_SOURCE _CONSENT_SOURCE = "python method" _CONSENT_OVERRIDE = ConsentStatus.DENIED _logger.warning("Telemetry is disabled for this session.") ``` # testing ### Test pipeline configuration package locally When you have created your inference pipeline package, you could go straight on with deploying it to the AI Inference Server. **However, we strongly recommend that you test your package before you deploy it.** The benefits of local testing are the following: - You can figure out many potential problems quicker, as you don't have to go through a deployment cycle. - You can diagnose and troubleshoot issues more easily, as you can inspect artifacts in your development environment. - You can validate your fixes quicker and move on to further issues that have not surfaced yet due to earlier issues. - You can easily include the local pipeline tests into the test automation in your build process. In general, we encourage you to apply state-of-the-art software engineering practices, such as unit testing and test driven development. This means that ideally you already have automated unit or even integration tests in place that make sure that the Python code and the saved models work according to expectations in isolation. This helps you localize errors when you put these pieces together and integrate them as a pipeline configuration package. AI SDK package `simaticai.testing` provides two tools for local testing: - A pipeline validator, that performs a static validation of the package concerning the availability of required Python packages. - A pipeline runner, that lets you simulate the execution of your pipeline in your Python environment. Please note that all this functionality applies to pipeline configuration packages, not edge configuration packages. In other words, you must use them before you convert your pipeline configuration package to an edge configuration package using the `convert_package` function. As the conversion itself is done in an automated way, most potential problems are already present in the package before the conversion, so a verification after conversion would only delay identifying these problems. For more comprehensive guidance on how to test pipelines before deployment, we recommend you refer to the AI SDK User Manual, especially the chapter concerning local testing of pipeline configuration packages. We also recommend you study the project templates for the AI SDK, which provide concrete code examples that show how to feed a pipeline with different kinds of inputs in a local test. ## `LocalPipelineRunner` Simulates the execution of a pipeline in a local Python environment. Restriction: only linear pipelines are supported where the pipeline input variables are only used by one component, each component uses only the outputs of the previous components, and the pipeline output only consists of variables from the last component. If the caller specifies no `path`, the working directory is temporary and is removed unless an error occurs. If the caller specifies a working directory with a `path` argument, the working directory is kept. This behavior can be overridden using boolean parameter `cleanup`. Currently, the pipeline runner supports both the current `process_input(data: dict)` entrypoint signature and the legacy `run(data: str)` signature. If both entrypoints are present, `process_input()` takes precedence. Please note however that `run()` is deprecated, and support for it will be removed in future versions of the pipeline runner. Parameters: | Name | Type | Description | Default | | ------------ | ------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- | | `packageZip` | `path - like` | Path to the pipeline configuration package. | *required* | | `path` | `path - like` | Path to the working directory. If unset, a temporary directory is created. | `None` | | `cleanup` | `bool` | If set, the working directory is kept when True, and deleted when False. If unset, a temporary working directory is removed, and an explicit working directory is kept. When an error occurs in a component, the working directory is kept regardless of this value. | `None` | Source code in `docs/industrial-ai-suite/sdk/simaticai/testing/pipeline_runner.py` ```python class LocalPipelineRunner: """ Simulates the execution of a pipeline in a local Python environment. Restriction: only linear pipelines are supported where the pipeline input variables are only used by one component, each component uses only the outputs of the previous components, and the pipeline output only consists of variables from the last component. If the caller specifies no `path`, the working directory is temporary and is removed unless an error occurs. If the caller specifies a working directory with a `path` argument, the working directory is kept. This behavior can be overridden using boolean parameter `cleanup`. Currently, the pipeline runner supports both the current `process_input(data: dict)` entrypoint signature and the legacy `run(data: str)` signature. If both entrypoints are present, `process_input()` takes precedence. Please note however that `run()` is deprecated, and support for it will be removed in future versions of the pipeline runner. Args: packageZip (path-like): Path to the pipeline configuration package. path (path-like): Path to the working directory. If unset, a temporary directory is created. cleanup (bool): If set, the working directory is kept when True, and deleted when False. \ If unset, a temporary working directory is removed, and an explicit working directory is kept. \ When an error occurs in a component, the working directory is kept regardless of this value. """ def __init__(self, packageZip: os.PathLike, path: Optional[os.PathLike] = None, cleanup: Optional[bool] = None, loglevel = logging.INFO): """ Creates a new component LocalPipelineRunner for the provided pipeline configuration package. Only works with a pipeline configuration package. Does not work with e.g. an edge configuration package. Args: packageZip (path-like): Path to the pipeline configuration package. path (path-like): Path to the working directory. If unset a temporary directory will be created. cleanup (bool): If set, the working directory will be kept when True, and deleted when False. \ If unset, a temporary working directory will be removed, and an explicit working directory will be kept. \ When an error occurs in a component, the working directory will be kept regardless of this value. """ self.package_zip: Path = Path(packageZip) self.path = Path(path).resolve().absolute() if path is not None else None self.components = {} self.parameters = {} self.cleanup = cleanup self.log_level = loglevel self.workdir: Path self.docker = VesselBaseDocker() self.report_writer = PipelineRunnerReportWriter() report_writer_handler = ReportWriterHandler(self.report_writer) _logger.addHandler(report_writer_handler) def __enter__(self): self.report_writer.set_package_zip_path(self.package_zip) timestamp = re.sub(r"[-:]", "", datetime.now(timezone.utc).isoformat(sep="_", timespec="seconds")) if self.path is not None: self.workdir = Path(self.path) self.workdir.mkdir(parents=True, exist_ok=True) self.cleanup = self.cleanup if self.cleanup is not None else False else: self.workdir = Path(tempfile.mkdtemp(prefix=f"LocalPipelineRunner_{timestamp}_")) self.cleanup = self.cleanup if self.cleanup is not None else True unzip_components = False with zipfile.ZipFile(self.package_zip) as zf: if 'runtime_config.yml' in zf.namelist(): self.workdir = self.workdir / self.package_zip.stem zf.extractall(path=self.workdir) unzip_components = True else: zf.extractall(path=self.workdir) self.workdir = self.workdir / zf.namelist()[0] try: with open(self.workdir / "pipeline_config.yml") as cf: config = yaml.load(cf, Loader=yaml.FullLoader) components = config["dataFlowPipeline"].get("components", []) for component in components: component["context"] = None component["env_dir"] = self.workdir / component['name'] self.components[component["name"]] = component if unzip_components: for component in components: component_zip = f"{component['name']}_{component['version']}.zip" with zipfile.ZipFile(self.workdir / 'components' / component_zip) as zf: zf.extractall(path=self.workdir / component["name"]) for parameter in config["dataFlowPipeline"].get("pipelineParameters", {}): self.parameters[parameter["name"]] = parameter except Exception: raise RuntimeError(INVALID_PIPELINE_PACKAGE_MESSAGE) return self def __exit__(self, exception_type, value, traceback): self.update_telemetry_data() self.report_writer.write_report() if self.cleanup: _logger.info("Removing local pipeline runner environment...") shutil.rmtree(self.workdir.parent) else: _logger.info(f"Leaving local pipeline runner environment in its final state at '{self.workdir}'") def collect_telemetry_data(self) -> dict: """ Collects telemetry data about the platform, environment, industrial AI packages, and pipeline. Returns: dict: A dictionary containing the telemetry data. """ telemetry_data = {} telemetry_data["platform"] = {} telemetry_data["platform"]["os"] = platform.system() telemetry_data["platform"]["release"] = platform.release() telemetry_data["platform"]["python_version"] = platform.python_version() _logger.info(f"locals: {locals()}") telemetry_data["environment"] = {} env_info = "Jupyter" if any(k for k in locals() if k in ["__IPYTHON__", "get_ipython"]) else MSG_NOT_FOUND env_info = "GitLab CI/CD" if "GITLAB_CI" in os.environ else env_info env_info = "Azure DevOps Pipelines" if "TF_BUILD" in os.environ else env_info env_info = "GitHub Actions" if "GITHUB_ACTIONS" in os.environ else env_info telemetry_data["environment"]["info"] = env_info telemetry_data["industrial_ai"] = {} try: telemetry_data["industrial_ai"]["simaticai"] = importlib_metadata.version("simaticai") except importlib_metadata.PackageNotFoundError: telemetry_data["industrial_ai"]["simaticai"] = MSG_NOT_FOUND telemetry_data["pipeline"] = {} telemetry_data["pipeline"]["python_versions"] = list(set(self.components[component]['runtime']['version'] for component in self.components if self.components[component]['runtime']['type'] == 'python')) telemetry_data["pipeline"]["file_extensions"] = [] for component_dir in [Path(self.workdir) / c for c in Path(self.workdir).rglob("*") if c.name in self.components.keys()]: excluded_dirs = set([component_dir / '.venv', component_dir / '__pyache__']) suffixes = list(set(f.suffix for f in component_dir.rglob("*") if not (any(excluded_dirs.intersection(f.parents)) or f.suffix in ["", ".zip", ".yml", ".yaml", ".html"]))) for suffix in suffixes: if suffix not in telemetry_data["pipeline"]["file_extensions"]: telemetry_data["pipeline"]["file_extensions"].append(suffix) return telemetry_data def update_telemetry_data(self): """ Update the telemetry data and the package. This method updates the telemetry data by loading the existing data from a YAML file, or collecting new telemetry data if the file doesn't exist. It then updates the "last_test_run" field of the telemetry data with the current timestamp. The updated telemetry data is then written back to the YAML file. If the package contains a different version of the telemetry data file, a new package is created with the updated telemetry data. Otherwise, the existing package is overwritten with the new package containing the updated telemetry data. """ _logger.info("Updating telemetry data and the package") telemetry_path = self.workdir / "telemetry_data.yml" if telemetry_path.is_file(): telemetry_data = yaml.safe_load(telemetry_path.read_text()) else: telemetry_data = self.collect_telemetry_data() telemetry_data["pipeline"]["last_test_run"] = datetime.now().isoformat() telemetry_path.write_text(yaml.dump(telemetry_data)) config_package = False with zipfile.ZipFile(self.package_zip, 'r') as zip_read: for file in zip_read.namelist(): if TELEMETRY_YAML in file and file != TELEMETRY_YAML: config_package = True break new_zip_path = Path(self.package_zip).parent / f"{Path(self.package_zip).stem}_tested.zip" with zipfile.ZipFile(self.package_zip, 'r') as zip_read: with zipfile.ZipFile(new_zip_path, 'w') as zip_write: for file in zip_read.namelist(): if TELEMETRY_YAML not in file: filepath = zip_read.extract(file, path=self.workdir) zip_write.write(filepath, arcname=file) else: zip_write.write(telemetry_path, arcname=file) if config_package: shutil.copy(new_zip_path, self.package_zip) else: new_sha_path = Path(self.package_zip).parent / f"{Path(self.package_zip).stem}_tested.sha256" new_sha_path.write_text(calc_sha(new_zip_path)) def _install_requirements(self, component, package_dir, no_index: bool = True): _logger.info("Installing requirements...") pip_report_file = component["env_dir"] / "pip_report.json" package_dir_path = _relative_to(package_dir, component["env_dir"]) pip_report_file_path = _relative_to(pip_report_file, component["env_dir"]) cmd = [ str(component['python_path']), "-m", "pip", "install", "--no-warn-script-location", "-f", f"{package_dir_path}", "-r", REQUIREMENTS_TXT, ] if no_index: cmd += [ "--no-index" ] use_no_deps = component.get("runtime", {}).get("installTransitiveDependencies", True) is False if use_no_deps: cmd += [ "--no-deps" ] result = subprocess.run(cmd, cwd=component["env_dir"], text=True, stderr=subprocess.PIPE) if 0 == result.returncode and use_no_deps: _logger.warning(f"Component `{component['name']}` is requested to be installed without transitive dependencies. " "This may lead to runtime errors if the dependencies are not handled manually.") # generate pip report with a dry run to get the list of installed packages (in case of CPU Python packages) if 0 == result.returncode and component.get("hwType", None) == "CPU": cmd += ["--dry-run", "--ignore-installed", "--report", f"{pip_report_file_path}"] subprocess.run(cmd, cwd=component["env_dir"], text=True, stderr=subprocess.PIPE) self.report_writer.add_installed_packages(component["name"], pip_report_file) return result def _install_from_packages_zip(self, component, package_dir): result = self._install_requirements(component, package_dir, True) return 0 == result.returncode def _install_from_pypi_org(self, component, package_dir): return self._install_requirements(component, package_dir, False) def _init_component_venv(self, component: dict): """ Creates a virtual environment in which the given component can run. Args: component (str): name of the selected component. """ _logger.info(f"Creating virtual environment for component '{component['name']}'...") context_dir = component["env_dir"] / ".venv" if self.docker.is_vessel: _logger.info("Creating virtual environment in docker...") component["context"], component["python_path"] = self.docker._create_venv(component["env_dir"], component["runtime"]["version"]) component["python_path"] = Path(component["python_path"]).resolve() _logger.info(f"Component context (docker): {component['context']}") else: builder = venv.EnvBuilder(with_pip=True, symlinks=False) builder.create(str(context_dir)) component["context"] = builder.ensure_directories(context_dir) component['python_path'] = Path(component["context"].env_exe).resolve() _logger.debug(f"Component python_path: {component['python_path']}") _logger.info("Upgrading pip...") cmd = [str(component['python_path']), "-m", "pip", "install", "pip", "--upgrade"] result = subprocess.run(cmd, cwd=component["env_dir"], text=True, stderr=subprocess.PIPE) if result.returncode != 0: self.cleanup = False _logger.warning(f"Error upgrading pip:\n{result.stderr}") cmd = [str(component['python_path']), "-m", "pip", "install", "--upgrade", "setuptools"] result = subprocess.run(cmd, cwd=component["env_dir"], text=True, stderr=subprocess.PIPE) if result.returncode != 0: self.cleanup = False _logger.warning(f"Error installing setuptools:\n{result.stderr}") try: result = self._install_logmodule(component["python_path"], component["env_dir"]) except Exception as err: _logger.error(err) self.cleanup = False raise RuntimeError("The 'simaticai' Python package is either not installed or does not contain package 'log_module'.") from None if result.returncode != 0: self.cleanup = False raise RuntimeError(f"Error installing log_module:\n{result.stderr}") req_list = Path(component["env_dir"] / "requirements.list") req_list.touch(exist_ok=True) if Path(component["env_dir"] / REQUIREMENTS_TXT).is_file(): dependencies, extra_index, index_url = parse_requirements(component["env_dir"] / REQUIREMENTS_TXT) requirements = "#".join(dependencies.keys()) req_list.write_text(requirements) else: _logger.info(f"'{REQUIREMENTS_TXT}' was not found. No additional dependencies were installed.") return package_dir = component["env_dir"] / PYTHON_PACKAGES package_zip = component["env_dir"] / PYTHON_PACKAGES_ZIP _logger.info(f"Extracting {PYTHON_PACKAGES_ZIP}") if package_zip.is_file(): with zipfile.ZipFile(package_zip) as zf: zf.extractall(path=package_dir.absolute()) else: _logger.info(f"There is no {PYTHON_PACKAGES_ZIP} to extract.") package_dir.mkdir(parents=True, exist_ok=True) success = self._install_from_packages_zip(component, package_dir) if not success: msg = f"Warning! Could not install dependencies from {PYTHON_PACKAGES_ZIP}. " msg += "Trying to install them from pypi.org. The resulting Python environment " msg += "may be significantly different than the targeted Python environment on the Edge Device!" _logger.warning(msg) if self.docker.is_vessel and component["runtime"]["type"] == "python": raise RuntimeError("The component is running in a docker container. The installation of dependencies from pypi.org is not supported.") else: second_install_result = self._install_from_pypi_org(component, package_dir) if 0 != second_install_result.returncode: self.cleanup = False raise RuntimeError(f"Error installing requirements:\n{second_install_result.stderr}") @staticmethod def _install_logmodule(python_path, env_dir): _logger.info("Installing LogModule...") try: package_paths = importlib_metadata.files("simaticai") assert package_paths is not None logger_wheel = [p for p in package_paths if 'log_module' in str(p)][0].locate() except Exception: from importlib.metadata import Distribution direct_url = Distribution.from_name("simaticai").read_text("direct_url.json") assert direct_url is not None direct_url = json.loads(direct_url)['url'] direct_url = direct_url.replace('file://','') direct_url = Path(direct_url) / 'simaticai' / 'data' paths = list(direct_url.rglob('*.whl')) logger_wheel = [p for p in paths if 'log_module' in str(p)][0].resolve() cmd = [ str(python_path), "-m", "pip", "install", "--no-warn-script-location", logger_wheel, "joblib" ] return subprocess.run(cmd, cwd=env_dir, text=True, stderr=subprocess.PIPE) def run_component(self, name: str, data: Optional[Union[dict, list, DataStream]]) -> Optional[Union[dict, list]]: """ Runs the component in its virtual environment with the given input. This environment is created according to `requirements.txt` in the package. Additionally 'joblib' and the mock `log_module` is automatically installed in this virtual environment. The input data can be a single input record in a dictionary or a batch of input records in a list of dictionaries, or a DataStream object which will produce the appropriate input data. The supplied input data is saved as `inputs.joblib` in the component runtime directory, and the output is saved as `output.joblib`. Args: name (str): The name of the component to be executed. data (dict or list): One or more input records for the component. Returns: dict / list: A list of dictionaries for outputs if there were no errors and field `ready` is true. If the input was a single dict, then a single dict (the first item of the list) or None if there is no output. """ assert name in self.components, f"Invalid component name: {name}" component = self.components[name] assert component["runtime"]["type"] in ["python", "gpuruntime"], f"Can not run component '{name}': Runtime type is nor 'python' or 'gpuruntime'" input_payload_path: Path = component["env_dir"] / "input.joblib" output_payload_path: Path = component["env_dir"] / "output.joblib" batch_input: bool = component["batch"]["inputBatch"] == "Yes" if component.get("batch") is not None else False batch_output: bool = component["batch"]["outputBatch"] == "Yes" if component.get("batch") is not None else False # Validate and serialize input payload assert data is not None, f"Can not run component '{name}' without input." result_is_list = True if isinstance(data, list): input_payload = data elif isinstance(data, DataStream): input_payload = [item for item in data] else: result_is_list = False input_payload = [data] validate_payload(input_payload, component["inputType"], batch_input) joblib.dump(input_payload, input_payload_path) self.report_writer.set_input_payload_length(name, len(input_payload)) _logger.info(f"Input payload saved as '{input_payload_path}'") # Assemble command for runnig component if component['runtime']['type'] == 'python': # Version check for Python e_major, e_minor, _, _, _ = sys.version_info c_major, c_minor, *_ = tuple(str(component["runtime"]["version"]).split('.')) if not self.docker.is_vessel and (f"{e_major}.{e_minor}" != f"{c_major}.{c_minor}"): msg = f"The local python version ({e_major}.{e_minor}) and the python version defined for the component ({c_major}.{c_minor}) are different." msg += " Testing will be done using dependencies that corresponds to the python version of your development environment." msg += " Pipeline behavior on AI Inference Server might be different." _logger.warning(msg) if component["context"] is None: self._init_component_venv(component) shutil.copy(_runner_path / 'run_component.py', component["env_dir"]) req_list_path = _relative_to(component["env_dir"] / "requirements.list", component["env_dir"]) json_params = json.dumps({param["name"]: param["defaultValue"] for param in self.parameters.values()}) args = [ "-m", 'run_component', "-m", Path(component["entrypoint"]).stem, "-p", f"{json_params}", "-r", f"{req_list_path}", ] else: # gpuruntime step requires Python environment with onnxruntime installed if component["context"] is None: shutil.copy(_runner_path / 'gpuruntime_requirements.txt', component["env_dir"] / REQUIREMENTS_TXT) self._init_component_venv(component) shutil.copy(_runner_path / 'run_gpuruntime_component.py', component["env_dir"]) shutil.copy(_runner_path.parent / 'model_config_pb2.py', component["env_dir"]) args = [ "-m", 'run_gpuruntime_component', "-m", component["entrypoint"], "-c", "config.pbtxt", ] args += [ "-i", input_payload_path.name, "-o", output_payload_path.name, "-ll", logging.getLevelName(self.log_level) ] # Run the component in the created Python environment _logger.info(f"Running component '{name}'...") _logger.info(f"{component['python_path'] =} to {component['env_dir'] =}") cmd = [str(component['python_path'])] + args _logger.info(f"Running command: {subprocess.list2cmdline(cmd)}") p = subprocess.Popen(cmd, cwd=component["env_dir"], stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True) if p.stdout is not None: for line in p.stdout: _logger.info(line.strip()) returncode = p.wait() if returncode not in [RETURN_CODE_OK, RETURN_CODE_DEPRECATED]: self.cleanup = False raise RuntimeError(f"""\ There was an error while running component '{name}'. You can check the test results in directory '{component['env_dir']}' """) else: response_wrapped = True if returncode == RETURN_CODE_DEPRECATED else False # Deserialize and validate output payload _logger.info(f"Loading output payload from '{output_payload_path}'") output_payload = joblib.load(output_payload_path) output_payload = output_payload if isinstance(output_payload, list) else [output_payload] self.report_writer.set_output_payload_length(name, len(output_payload)) if response_wrapped: output_payload = [json.loads(item['output']) for item in output_payload if item['ready']] else: output_payload = [output for output in output_payload if output is not None] validate_payload(output_payload, component["outputType"], batch_output) if result_is_list: return output_payload if 0 < len(output_payload): return output_payload[0] return None def update_parameters(self, parameters: dict): """ Validates and updates pipeline parameters. The elements of the dictionary must match the parameters specified in the pipeline configuration package. If any of the names or types does not match, all parameters will remain untouched. Args: parameters (dict): names and values of parameters to update Raises: AssertionError: When: - either `name` is not in the configured parameters, - or `defaultValue` type is different from the configured one """ for key,value in parameters.items(): if key not in self.parameters.keys(): if key.upper().startswith("__AI_IS"): self.parameters[key] = {'defaultValue': False, 'name': key, 'type': 'Boolean'} else: raise AssertionError(f"Pipeline has no parameters with the name '{key}' and type '{type_map.get(type(value).__name__)}'") elif self.parameters[key]["type"] != type_map.get(type(value).__name__): raise AssertionError(f"Pipeline has no parameters with the name '{key}' and type '{type_map.get(type(value).__name__)}'") for key, value in parameters.items(): self.parameters[key]["defaultValue"] = value def run_pipeline(self, payload: Optional[Union[dict, list, DataStream]] = {}) -> Optional[Union[dict, list]]: """ Runs all the components sequentially, assuming the output of a component is only consumed by the next. The input data can be a single input record in a dictionary or a batch of input records in a list of dictionaries. For each component the supplied input data is saved as `input.joblib` in the component runtime directory, and the output is saved as `output.joblib`. Args: payload (dict or list): One or more input records for the pipeline. Returns: The output of the last component. """ for name in self.components.keys(): payload = self.run_component(name, payload) return payload ``` ### `__init__(packageZip, path=None, cleanup=None, loglevel=logging.INFO)` Creates a new component LocalPipelineRunner for the provided pipeline configuration package. Only works with a pipeline configuration package. Does not work with e.g. an edge configuration package. Parameters: | Name | Type | Description | Default | | ------------ | ------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- | | `packageZip` | `path - like` | Path to the pipeline configuration package. | *required* | | `path` | `path - like` | Path to the working directory. If unset a temporary directory will be created. | `None` | | `cleanup` | `bool` | If set, the working directory will be kept when True, and deleted when False. If unset, a temporary working directory will be removed, and an explicit working directory will be kept. When an error occurs in a component, the working directory will be kept regardless of this value. | `None` | Source code in `docs/industrial-ai-suite/sdk/simaticai/testing/pipeline_runner.py` ```python def __init__(self, packageZip: os.PathLike, path: Optional[os.PathLike] = None, cleanup: Optional[bool] = None, loglevel = logging.INFO): """ Creates a new component LocalPipelineRunner for the provided pipeline configuration package. Only works with a pipeline configuration package. Does not work with e.g. an edge configuration package. Args: packageZip (path-like): Path to the pipeline configuration package. path (path-like): Path to the working directory. If unset a temporary directory will be created. cleanup (bool): If set, the working directory will be kept when True, and deleted when False. \ If unset, a temporary working directory will be removed, and an explicit working directory will be kept. \ When an error occurs in a component, the working directory will be kept regardless of this value. """ self.package_zip: Path = Path(packageZip) self.path = Path(path).resolve().absolute() if path is not None else None self.components = {} self.parameters = {} self.cleanup = cleanup self.log_level = loglevel self.workdir: Path self.docker = VesselBaseDocker() self.report_writer = PipelineRunnerReportWriter() report_writer_handler = ReportWriterHandler(self.report_writer) _logger.addHandler(report_writer_handler) ``` ### `collect_telemetry_data()` Collects telemetry data about the platform, environment, industrial AI packages, and pipeline. Returns: | Name | Type | Description | | ------ | ------ | ------------------------------------------- | | `dict` | `dict` | A dictionary containing the telemetry data. | Source code in `docs/industrial-ai-suite/sdk/simaticai/testing/pipeline_runner.py` ```python def collect_telemetry_data(self) -> dict: """ Collects telemetry data about the platform, environment, industrial AI packages, and pipeline. Returns: dict: A dictionary containing the telemetry data. """ telemetry_data = {} telemetry_data["platform"] = {} telemetry_data["platform"]["os"] = platform.system() telemetry_data["platform"]["release"] = platform.release() telemetry_data["platform"]["python_version"] = platform.python_version() _logger.info(f"locals: {locals()}") telemetry_data["environment"] = {} env_info = "Jupyter" if any(k for k in locals() if k in ["__IPYTHON__", "get_ipython"]) else MSG_NOT_FOUND env_info = "GitLab CI/CD" if "GITLAB_CI" in os.environ else env_info env_info = "Azure DevOps Pipelines" if "TF_BUILD" in os.environ else env_info env_info = "GitHub Actions" if "GITHUB_ACTIONS" in os.environ else env_info telemetry_data["environment"]["info"] = env_info telemetry_data["industrial_ai"] = {} try: telemetry_data["industrial_ai"]["simaticai"] = importlib_metadata.version("simaticai") except importlib_metadata.PackageNotFoundError: telemetry_data["industrial_ai"]["simaticai"] = MSG_NOT_FOUND telemetry_data["pipeline"] = {} telemetry_data["pipeline"]["python_versions"] = list(set(self.components[component]['runtime']['version'] for component in self.components if self.components[component]['runtime']['type'] == 'python')) telemetry_data["pipeline"]["file_extensions"] = [] for component_dir in [Path(self.workdir) / c for c in Path(self.workdir).rglob("*") if c.name in self.components.keys()]: excluded_dirs = set([component_dir / '.venv', component_dir / '__pyache__']) suffixes = list(set(f.suffix for f in component_dir.rglob("*") if not (any(excluded_dirs.intersection(f.parents)) or f.suffix in ["", ".zip", ".yml", ".yaml", ".html"]))) for suffix in suffixes: if suffix not in telemetry_data["pipeline"]["file_extensions"]: telemetry_data["pipeline"]["file_extensions"].append(suffix) return telemetry_data ``` ### `update_telemetry_data()` Update the telemetry data and the package. This method updates the telemetry data by loading the existing data from a YAML file, or collecting new telemetry data if the file doesn't exist. It then updates the "last_test_run" field of the telemetry data with the current timestamp. The updated telemetry data is then written back to the YAML file. If the package contains a different version of the telemetry data file, a new package is created with the updated telemetry data. Otherwise, the existing package is overwritten with the new package containing the updated telemetry data. Source code in `docs/industrial-ai-suite/sdk/simaticai/testing/pipeline_runner.py` ```python def update_telemetry_data(self): """ Update the telemetry data and the package. This method updates the telemetry data by loading the existing data from a YAML file, or collecting new telemetry data if the file doesn't exist. It then updates the "last_test_run" field of the telemetry data with the current timestamp. The updated telemetry data is then written back to the YAML file. If the package contains a different version of the telemetry data file, a new package is created with the updated telemetry data. Otherwise, the existing package is overwritten with the new package containing the updated telemetry data. """ _logger.info("Updating telemetry data and the package") telemetry_path = self.workdir / "telemetry_data.yml" if telemetry_path.is_file(): telemetry_data = yaml.safe_load(telemetry_path.read_text()) else: telemetry_data = self.collect_telemetry_data() telemetry_data["pipeline"]["last_test_run"] = datetime.now().isoformat() telemetry_path.write_text(yaml.dump(telemetry_data)) config_package = False with zipfile.ZipFile(self.package_zip, 'r') as zip_read: for file in zip_read.namelist(): if TELEMETRY_YAML in file and file != TELEMETRY_YAML: config_package = True break new_zip_path = Path(self.package_zip).parent / f"{Path(self.package_zip).stem}_tested.zip" with zipfile.ZipFile(self.package_zip, 'r') as zip_read: with zipfile.ZipFile(new_zip_path, 'w') as zip_write: for file in zip_read.namelist(): if TELEMETRY_YAML not in file: filepath = zip_read.extract(file, path=self.workdir) zip_write.write(filepath, arcname=file) else: zip_write.write(telemetry_path, arcname=file) if config_package: shutil.copy(new_zip_path, self.package_zip) else: new_sha_path = Path(self.package_zip).parent / f"{Path(self.package_zip).stem}_tested.sha256" new_sha_path.write_text(calc_sha(new_zip_path)) ``` ### `run_component(name, data)` Runs the component in its virtual environment with the given input. This environment is created according to `requirements.txt` in the package. Additionally 'joblib' and the mock `log_module` is automatically installed in this virtual environment. The input data can be a single input record in a dictionary or a batch of input records in a list of dictionaries, or a DataStream object which will produce the appropriate input data. The supplied input data is saved as `inputs.joblib` in the component runtime directory, and the output is saved as `output.joblib`. Parameters: | Name | Type | Description | Default | | ------ | -------------- | -------------------------------------------- | ---------- | | `name` | `str` | The name of the component to be executed. | *required* | | `data` | `dict or list` | One or more input records for the component. | *required* | Returns: | Type | Description | | ----------------------------- | -------------------------------------------------------------------------------------------------------------- | | `Optional[Union[dict, list]]` | dict / list: A list of dictionaries for outputs if there were no errors and field ready is true. | | `Optional[Union[dict, list]]` | If the input was a single dict, then a single dict (the first item of the list) or None if there is no output. | Source code in `docs/industrial-ai-suite/sdk/simaticai/testing/pipeline_runner.py` ```python def run_component(self, name: str, data: Optional[Union[dict, list, DataStream]]) -> Optional[Union[dict, list]]: """ Runs the component in its virtual environment with the given input. This environment is created according to `requirements.txt` in the package. Additionally 'joblib' and the mock `log_module` is automatically installed in this virtual environment. The input data can be a single input record in a dictionary or a batch of input records in a list of dictionaries, or a DataStream object which will produce the appropriate input data. The supplied input data is saved as `inputs.joblib` in the component runtime directory, and the output is saved as `output.joblib`. Args: name (str): The name of the component to be executed. data (dict or list): One or more input records for the component. Returns: dict / list: A list of dictionaries for outputs if there were no errors and field `ready` is true. If the input was a single dict, then a single dict (the first item of the list) or None if there is no output. """ assert name in self.components, f"Invalid component name: {name}" component = self.components[name] assert component["runtime"]["type"] in ["python", "gpuruntime"], f"Can not run component '{name}': Runtime type is nor 'python' or 'gpuruntime'" input_payload_path: Path = component["env_dir"] / "input.joblib" output_payload_path: Path = component["env_dir"] / "output.joblib" batch_input: bool = component["batch"]["inputBatch"] == "Yes" if component.get("batch") is not None else False batch_output: bool = component["batch"]["outputBatch"] == "Yes" if component.get("batch") is not None else False # Validate and serialize input payload assert data is not None, f"Can not run component '{name}' without input." result_is_list = True if isinstance(data, list): input_payload = data elif isinstance(data, DataStream): input_payload = [item for item in data] else: result_is_list = False input_payload = [data] validate_payload(input_payload, component["inputType"], batch_input) joblib.dump(input_payload, input_payload_path) self.report_writer.set_input_payload_length(name, len(input_payload)) _logger.info(f"Input payload saved as '{input_payload_path}'") # Assemble command for runnig component if component['runtime']['type'] == 'python': # Version check for Python e_major, e_minor, _, _, _ = sys.version_info c_major, c_minor, *_ = tuple(str(component["runtime"]["version"]).split('.')) if not self.docker.is_vessel and (f"{e_major}.{e_minor}" != f"{c_major}.{c_minor}"): msg = f"The local python version ({e_major}.{e_minor}) and the python version defined for the component ({c_major}.{c_minor}) are different." msg += " Testing will be done using dependencies that corresponds to the python version of your development environment." msg += " Pipeline behavior on AI Inference Server might be different." _logger.warning(msg) if component["context"] is None: self._init_component_venv(component) shutil.copy(_runner_path / 'run_component.py', component["env_dir"]) req_list_path = _relative_to(component["env_dir"] / "requirements.list", component["env_dir"]) json_params = json.dumps({param["name"]: param["defaultValue"] for param in self.parameters.values()}) args = [ "-m", 'run_component', "-m", Path(component["entrypoint"]).stem, "-p", f"{json_params}", "-r", f"{req_list_path}", ] else: # gpuruntime step requires Python environment with onnxruntime installed if component["context"] is None: shutil.copy(_runner_path / 'gpuruntime_requirements.txt', component["env_dir"] / REQUIREMENTS_TXT) self._init_component_venv(component) shutil.copy(_runner_path / 'run_gpuruntime_component.py', component["env_dir"]) shutil.copy(_runner_path.parent / 'model_config_pb2.py', component["env_dir"]) args = [ "-m", 'run_gpuruntime_component', "-m", component["entrypoint"], "-c", "config.pbtxt", ] args += [ "-i", input_payload_path.name, "-o", output_payload_path.name, "-ll", logging.getLevelName(self.log_level) ] # Run the component in the created Python environment _logger.info(f"Running component '{name}'...") _logger.info(f"{component['python_path'] =} to {component['env_dir'] =}") cmd = [str(component['python_path'])] + args _logger.info(f"Running command: {subprocess.list2cmdline(cmd)}") p = subprocess.Popen(cmd, cwd=component["env_dir"], stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True) if p.stdout is not None: for line in p.stdout: _logger.info(line.strip()) returncode = p.wait() if returncode not in [RETURN_CODE_OK, RETURN_CODE_DEPRECATED]: self.cleanup = False raise RuntimeError(f"""\ There was an error while running component '{name}'. You can check the test results in directory '{component['env_dir']}' """) else: response_wrapped = True if returncode == RETURN_CODE_DEPRECATED else False # Deserialize and validate output payload _logger.info(f"Loading output payload from '{output_payload_path}'") output_payload = joblib.load(output_payload_path) output_payload = output_payload if isinstance(output_payload, list) else [output_payload] self.report_writer.set_output_payload_length(name, len(output_payload)) if response_wrapped: output_payload = [json.loads(item['output']) for item in output_payload if item['ready']] else: output_payload = [output for output in output_payload if output is not None] validate_payload(output_payload, component["outputType"], batch_output) if result_is_list: return output_payload if 0 < len(output_payload): return output_payload[0] return None ``` ### `update_parameters(parameters)` Validates and updates pipeline parameters. The elements of the dictionary must match the parameters specified in the pipeline configuration package. If any of the names or types does not match, all parameters will remain untouched. Args: ```text parameters (dict): names and values of parameters to update ``` Raises: ```text AssertionError: When: - either `name` is not in the configured parameters, - or `defaultValue` type is different from the configured one ``` Source code in `docs/industrial-ai-suite/sdk/simaticai/testing/pipeline_runner.py` ```python def update_parameters(self, parameters: dict): """ Validates and updates pipeline parameters. The elements of the dictionary must match the parameters specified in the pipeline configuration package. If any of the names or types does not match, all parameters will remain untouched. Args: parameters (dict): names and values of parameters to update Raises: AssertionError: When: - either `name` is not in the configured parameters, - or `defaultValue` type is different from the configured one """ for key,value in parameters.items(): if key not in self.parameters.keys(): if key.upper().startswith("__AI_IS"): self.parameters[key] = {'defaultValue': False, 'name': key, 'type': 'Boolean'} else: raise AssertionError(f"Pipeline has no parameters with the name '{key}' and type '{type_map.get(type(value).__name__)}'") elif self.parameters[key]["type"] != type_map.get(type(value).__name__): raise AssertionError(f"Pipeline has no parameters with the name '{key}' and type '{type_map.get(type(value).__name__)}'") for key, value in parameters.items(): self.parameters[key]["defaultValue"] = value ``` ### `run_pipeline(payload={})` Runs all the components sequentially, assuming the output of a component is only consumed by the next. The input data can be a single input record in a dictionary or a batch of input records in a list of dictionaries. For each component the supplied input data is saved as `input.joblib` in the component runtime directory, and the output is saved as `output.joblib`. Parameters: | Name | Type | Description | Default | | --------- | -------------- | ------------------------------------------- | ------- | | `payload` | `dict or list` | One or more input records for the pipeline. | `{}` | Returns: | Type | Description | | ----------------------------- | --------------------------------- | | `Optional[Union[dict, list]]` | The output of the last component. | Source code in `docs/industrial-ai-suite/sdk/simaticai/testing/pipeline_runner.py` ```python def run_pipeline(self, payload: Optional[Union[dict, list, DataStream]] = {}) -> Optional[Union[dict, list]]: """ Runs all the components sequentially, assuming the output of a component is only consumed by the next. The input data can be a single input record in a dictionary or a batch of input records in a list of dictionaries. For each component the supplied input data is saved as `input.joblib` in the component runtime directory, and the output is saved as `output.joblib`. Args: payload (dict or list): One or more input records for the pipeline. Returns: The output of the last component. """ for name in self.components.keys(): payload = self.run_component(name, payload) return payload ``` ## `ComponentRunner` Class to run a Pipeline Component in a virtual environment. Supported Component Types: - PythonComponent - GPURuntimeComponent Args: component: PythonComponent or GPURuntimeComponent workdir: Path to the directory where the component should be run. If None, the current working directory is used. cleanup: If True, the workdir is deleted after the context manager exits. Source code in `docs/industrial-ai-suite/sdk/simaticai/testing/component_runner.py` ```python class ComponentRunner(): """ Class to run a Pipeline Component in a virtual environment. Supported Component Types: - PythonComponent - GPURuntimeComponent Args: component: PythonComponent or GPURuntimeComponent workdir: Path to the directory where the component should be run. If None, the current working directory is used. cleanup: If True, the workdir is deleted after the context manager exits. """ def __init__(self, component, workdir=None, cleanup=False): self.component = component self.parameters = { '__AI_IS_IMAGE_SET_VISUALIZATION': False } self.cleanup = cleanup self._logger = self._set_logger() self._create_workdir(component, workdir) self._copy_resources() self._create_venv() self._install_requirements() def __enter__(self): return self def __exit__(self, exception_type, exception_value, traceback): if exception_type is not None: self._logger.error(f"Exception occurred: {exception_type.__name__}: {exception_value}") return False if self.cleanup: shutil.rmtree(self.workdir) def _set_logger(self): """ Set the logger for the ComponentRunner. """ logger = logging.getLogger(__name__) log_level = os.environ.get("loglevel", "INFO").upper() logger.setLevel(log_level) handler = logging.StreamHandler() handler.setLevel(log_level) formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s') handler.setFormatter(formatter) logger.addHandler(handler) return logger def _create_workdir(self, component, workdir): """ Create a workdir for the component. """ if workdir is None: self.workdir = Path.cwd() / component.name else: self.workdir = Path(workdir).resolve().absolute() / component.name self.workdir.mkdir(parents=True, exist_ok=True) self._logger.info(f"Created workdir: {self.workdir}") def _copy_resources(self): """ Copy the required resources of the component to the workdir. In case of a PythonComponent, also creates the dependencies into requirements.txt. In case of a GPURuntimeComponent, also creates a requirements.txt with the required onnx and onnxruntime dependencies. Also adds the runner script to the workdir. In case of the source folder same as the workdir, the resources are not copied, so any changes in source will affect the workdir. """ if isinstance(self.component, PythonComponent): same_resources = [] for from_path in self.component.resources.keys(): try: to_path = self.workdir / self.component.resources[from_path] to_path.mkdir(parents=True, exist_ok=True) shutil.copy(from_path, to_path) except shutil.SameFileError: same_resources.append(from_path.name) if same_resources: self._logger.info("Resources are already in workdir, which are not copied:") for resource in same_resources: self._logger.debug(f" - {resource}") # download manually added Python packages and requirements.txt self.component.python_dependencies.save(self.workdir) shutil.copy(PYTHON_RUNNER_PATH, self.workdir) else: model_dir = Path(self.workdir / "1") model_dir.mkdir(parents=True, exist_ok=True) shutil.copy(self.component.model_path, model_dir / "model.onnx") Path(self.workdir / "config.pbtxt").write_text(f"{self.component.auto_config}") shutil.copy(GPU_RUNNER_PATH, self.workdir / "run_component.py") shutil.copy(GPU_REQUIREMENTS_PATH, self.workdir / "requirements.txt") shutil.copy(GPU_CONFIG_PATH, self.workdir) self._logger.info("Resources are copied into the workdir.") def _create_venv(self): """ Create a Python virtual environment in the workdir. The created virtual environment is stored in the context_dir and has the same version as the current Python interpreter. """ context_dir = self.workdir / ".venv" builder = venv.EnvBuilder(with_pip=True, symlinks=False) builder.create(context_dir) self.context = builder.ensure_directories(context_dir) self.python_path = Path(self.context.env_exe).resolve() self._logger.info(f"Python virtualenv created in folder '{self.context.env_dir}'") def _install_requirements(self): """ Installs the required Python dependencies. """ cmd = [str(self.python_path), '-m', 'pip', 'install', 'joblib', '-r', 'requirements.txt'] result = subprocess.run(cmd, cwd=self.workdir, capture_output=True) if result.returncode != 0: raise RuntimeError(result.stderr.decode()) result_lines = result.stdout.decode().split("\n") installed_message = " ".join([line for line in result_lines if "Successfully installed" in line]) self._logger.info(installed_message) def set_parameters(self, parameter_name, parameter_value): """ Set the parameters for the component. """ self.parameters[parameter_name] = parameter_value def run(self, input_payload: Optional[Union[dict, list]]) -> Optional[Union[dict, list]]: """ Run the component with the input payload. Parameters: input_payload: Input payload for the component. Returns: Output payload from the component. Side Effects: - The input payload is saved in the workdir. - The output payload is loaded from the workdir. """ input_payload_path: Path = self.workdir / "input.joblib" output_payload_path: Path = self.workdir / "output.joblib" batch_input: bool = self.component.batch.inputBatch if isinstance(input_payload, DataStream): input_payload = [item for item in input_payload] elif isinstance(input_payload, dict): input_payload = [input_payload] input_variables = [{'name': name, 'type': c_input['type']} for name, c_input in self.component.inputs.items()] self._validate_payload(input_payload, input_variables, batch_input) joblib.dump(input_payload, input_payload_path) cmd = self._create_command(input_payload_path, output_payload_path) self._logger.info("Running command: " + " ".join(cmd)) result = subprocess.run(cmd, cwd=self.workdir, capture_output=True) if result.returncode != 0: raise RuntimeError(result.stderr.decode()) self._logger.debug(result.stderr.decode()) output_payload = joblib.load(output_payload_path) batch_output: bool = self.component.batch.outputBatch output_payload = output_payload if isinstance(output_payload, list) else [output_payload] output_payload = [output for output in output_payload if output is not None] output_variables = [{'name': name, 'type': c_output['type']} for name, c_output in self.component.outputs.items()] if isinstance(self.component, PythonComponent): output_variables += [{'name': name, 'type': 'String'} for name in self.component.metrics] self._validate_payload(output_payload, output_variables, batch_output) return output_payload def _check_instance(self, element, element_name: str, instance): if not isinstance(element, instance): self._logger.error(f"{element_name} must be an instance of {instance.__name__}") raise ValueError(f"{element_name} must be an instance of {instance.__name__}") def _check_variable_types(self, variable: dict, value): if variable["type"] == "String": self._check_instance(value, "String value", str) if variable["type"] == "StringArray": self._check_instance(value, "StringArray value", list) for i in value: self._check_instance(i, "StringArray item", str) if variable["type"] == "Object": self._check_instance(value, "Object value", dict) values = list(value.values()) if len(values) != 2: self._logger.error("Object value must have exactly 2 items") raise ValueError("Object value must have exactly 2 items") ok = isinstance(values[0], str) and isinstance(values[1], bytes) or isinstance(values[1], str) and isinstance(values[0], bytes) if not ok: self._logger.error("Object value must have exactly one 'str' and one 'bytes' field") raise ValueError("Object value must have exactly one 'str' and one 'bytes' field") def _validate_payload(self, payload: list, variables: list, batch: bool): self._check_instance(payload, "Payload", list) for payload_element in payload: if batch: self._check_instance(payload_element, "Batch payload element", list) else: payload_element = [payload_element] for item in payload_element: self._validate_payload_item(item, variables) def _validate_payload_item(self, item: dict, variables: list): self._check_instance(item, "Payload item", dict) for variable in variables: name = variable["name"] value = item.get(name, None) if value is None: self._logger.warning(f"WARNING! Variable '{name}' is missing from input, output or metric") continue self._check_variable_types(variable, value) payload_names = set(item.keys()) variable_names = { variable["name"] for variable in variables } variable_names.add('timestamp') extra_variables = payload_names - variable_names if len(extra_variables): self._logger.warning(f"WARNING! These variables are not declared but are part of the payload: {extra_variables}") def _create_command(self, input_payload_path, output_payload_path): """ Create the command to run the component. """ cmd = [str(self.python_path), "-m", 'run_component'] if isinstance(self.component, PythonComponent): cmd += [ "-m", Path(self.component.entrypoint).stem, "-p", json.dumps(self.parameters) ] if isinstance(self.component, GPURuntimeComponent): model_path = Path(self.workdir / "1" / "model.onnx").absolute().resolve() config_path = Path(self.workdir / "config.pbtxt").absolute().resolve() cmd += [ "-m", str(model_path), "-c", str(config_path), ] cmd += [ "-i", input_payload_path.name, "-o", output_payload_path.name, "-ll", logging.getLevelName(self._logger.getEffectiveLevel()) ] return cmd ``` ### `set_parameters(parameter_name, parameter_value)` Set the parameters for the component. Source code in `docs/industrial-ai-suite/sdk/simaticai/testing/component_runner.py` ```python def set_parameters(self, parameter_name, parameter_value): """ Set the parameters for the component. """ self.parameters[parameter_name] = parameter_value ``` ### `run(input_payload)` Run the component with the input payload. Parameters: input_payload: Input payload for the component. Returns: Output payload from the component. Side Effects: - The input payload is saved in the workdir. - The output payload is loaded from the workdir. Source code in `docs/industrial-ai-suite/sdk/simaticai/testing/component_runner.py` ```python def run(self, input_payload: Optional[Union[dict, list]]) -> Optional[Union[dict, list]]: """ Run the component with the input payload. Parameters: input_payload: Input payload for the component. Returns: Output payload from the component. Side Effects: - The input payload is saved in the workdir. - The output payload is loaded from the workdir. """ input_payload_path: Path = self.workdir / "input.joblib" output_payload_path: Path = self.workdir / "output.joblib" batch_input: bool = self.component.batch.inputBatch if isinstance(input_payload, DataStream): input_payload = [item for item in input_payload] elif isinstance(input_payload, dict): input_payload = [input_payload] input_variables = [{'name': name, 'type': c_input['type']} for name, c_input in self.component.inputs.items()] self._validate_payload(input_payload, input_variables, batch_input) joblib.dump(input_payload, input_payload_path) cmd = self._create_command(input_payload_path, output_payload_path) self._logger.info("Running command: " + " ".join(cmd)) result = subprocess.run(cmd, cwd=self.workdir, capture_output=True) if result.returncode != 0: raise RuntimeError(result.stderr.decode()) self._logger.debug(result.stderr.decode()) output_payload = joblib.load(output_payload_path) batch_output: bool = self.component.batch.outputBatch output_payload = output_payload if isinstance(output_payload, list) else [output_payload] output_payload = [output for output in output_payload if output is not None] output_variables = [{'name': name, 'type': c_output['type']} for name, c_output in self.component.outputs.items()] if isinstance(self.component, PythonComponent): output_variables += [{'name': name, 'type': 'String'} for name in self.component.metrics] self._validate_payload(output_payload, output_variables, batch_output) return output_payload ``` ## `ComponentRunner` Class to run a Pipeline Component in a virtual environment. Supported Component Types: - PythonComponent - GPURuntimeComponent Args: component: PythonComponent or GPURuntimeComponent workdir: Path to the directory where the component should be run. If None, the current working directory is used. cleanup: If True, the workdir is deleted after the context manager exits. Source code in `docs/industrial-ai-suite/sdk/simaticai/testing/component_runner.py` ```python class ComponentRunner(): """ Class to run a Pipeline Component in a virtual environment. Supported Component Types: - PythonComponent - GPURuntimeComponent Args: component: PythonComponent or GPURuntimeComponent workdir: Path to the directory where the component should be run. If None, the current working directory is used. cleanup: If True, the workdir is deleted after the context manager exits. """ def __init__(self, component, workdir=None, cleanup=False): self.component = component self.parameters = { '__AI_IS_IMAGE_SET_VISUALIZATION': False } self.cleanup = cleanup self._logger = self._set_logger() self._create_workdir(component, workdir) self._copy_resources() self._create_venv() self._install_requirements() def __enter__(self): return self def __exit__(self, exception_type, exception_value, traceback): if exception_type is not None: self._logger.error(f"Exception occurred: {exception_type.__name__}: {exception_value}") return False if self.cleanup: shutil.rmtree(self.workdir) def _set_logger(self): """ Set the logger for the ComponentRunner. """ logger = logging.getLogger(__name__) log_level = os.environ.get("loglevel", "INFO").upper() logger.setLevel(log_level) handler = logging.StreamHandler() handler.setLevel(log_level) formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s') handler.setFormatter(formatter) logger.addHandler(handler) return logger def _create_workdir(self, component, workdir): """ Create a workdir for the component. """ if workdir is None: self.workdir = Path.cwd() / component.name else: self.workdir = Path(workdir).resolve().absolute() / component.name self.workdir.mkdir(parents=True, exist_ok=True) self._logger.info(f"Created workdir: {self.workdir}") def _copy_resources(self): """ Copy the required resources of the component to the workdir. In case of a PythonComponent, also creates the dependencies into requirements.txt. In case of a GPURuntimeComponent, also creates a requirements.txt with the required onnx and onnxruntime dependencies. Also adds the runner script to the workdir. In case of the source folder same as the workdir, the resources are not copied, so any changes in source will affect the workdir. """ if isinstance(self.component, PythonComponent): same_resources = [] for from_path in self.component.resources.keys(): try: to_path = self.workdir / self.component.resources[from_path] to_path.mkdir(parents=True, exist_ok=True) shutil.copy(from_path, to_path) except shutil.SameFileError: same_resources.append(from_path.name) if same_resources: self._logger.info("Resources are already in workdir, which are not copied:") for resource in same_resources: self._logger.debug(f" - {resource}") # download manually added Python packages and requirements.txt self.component.python_dependencies.save(self.workdir) shutil.copy(PYTHON_RUNNER_PATH, self.workdir) else: model_dir = Path(self.workdir / "1") model_dir.mkdir(parents=True, exist_ok=True) shutil.copy(self.component.model_path, model_dir / "model.onnx") Path(self.workdir / "config.pbtxt").write_text(f"{self.component.auto_config}") shutil.copy(GPU_RUNNER_PATH, self.workdir / "run_component.py") shutil.copy(GPU_REQUIREMENTS_PATH, self.workdir / "requirements.txt") shutil.copy(GPU_CONFIG_PATH, self.workdir) self._logger.info("Resources are copied into the workdir.") def _create_venv(self): """ Create a Python virtual environment in the workdir. The created virtual environment is stored in the context_dir and has the same version as the current Python interpreter. """ context_dir = self.workdir / ".venv" builder = venv.EnvBuilder(with_pip=True, symlinks=False) builder.create(context_dir) self.context = builder.ensure_directories(context_dir) self.python_path = Path(self.context.env_exe).resolve() self._logger.info(f"Python virtualenv created in folder '{self.context.env_dir}'") def _install_requirements(self): """ Installs the required Python dependencies. """ cmd = [str(self.python_path), '-m', 'pip', 'install', 'joblib', '-r', 'requirements.txt'] result = subprocess.run(cmd, cwd=self.workdir, capture_output=True) if result.returncode != 0: raise RuntimeError(result.stderr.decode()) result_lines = result.stdout.decode().split("\n") installed_message = " ".join([line for line in result_lines if "Successfully installed" in line]) self._logger.info(installed_message) def set_parameters(self, parameter_name, parameter_value): """ Set the parameters for the component. """ self.parameters[parameter_name] = parameter_value def run(self, input_payload: Optional[Union[dict, list]]) -> Optional[Union[dict, list]]: """ Run the component with the input payload. Parameters: input_payload: Input payload for the component. Returns: Output payload from the component. Side Effects: - The input payload is saved in the workdir. - The output payload is loaded from the workdir. """ input_payload_path: Path = self.workdir / "input.joblib" output_payload_path: Path = self.workdir / "output.joblib" batch_input: bool = self.component.batch.inputBatch if isinstance(input_payload, DataStream): input_payload = [item for item in input_payload] elif isinstance(input_payload, dict): input_payload = [input_payload] input_variables = [{'name': name, 'type': c_input['type']} for name, c_input in self.component.inputs.items()] self._validate_payload(input_payload, input_variables, batch_input) joblib.dump(input_payload, input_payload_path) cmd = self._create_command(input_payload_path, output_payload_path) self._logger.info("Running command: " + " ".join(cmd)) result = subprocess.run(cmd, cwd=self.workdir, capture_output=True) if result.returncode != 0: raise RuntimeError(result.stderr.decode()) self._logger.debug(result.stderr.decode()) output_payload = joblib.load(output_payload_path) batch_output: bool = self.component.batch.outputBatch output_payload = output_payload if isinstance(output_payload, list) else [output_payload] output_payload = [output for output in output_payload if output is not None] output_variables = [{'name': name, 'type': c_output['type']} for name, c_output in self.component.outputs.items()] if isinstance(self.component, PythonComponent): output_variables += [{'name': name, 'type': 'String'} for name in self.component.metrics] self._validate_payload(output_payload, output_variables, batch_output) return output_payload def _check_instance(self, element, element_name: str, instance): if not isinstance(element, instance): self._logger.error(f"{element_name} must be an instance of {instance.__name__}") raise ValueError(f"{element_name} must be an instance of {instance.__name__}") def _check_variable_types(self, variable: dict, value): if variable["type"] == "String": self._check_instance(value, "String value", str) if variable["type"] == "StringArray": self._check_instance(value, "StringArray value", list) for i in value: self._check_instance(i, "StringArray item", str) if variable["type"] == "Object": self._check_instance(value, "Object value", dict) values = list(value.values()) if len(values) != 2: self._logger.error("Object value must have exactly 2 items") raise ValueError("Object value must have exactly 2 items") ok = isinstance(values[0], str) and isinstance(values[1], bytes) or isinstance(values[1], str) and isinstance(values[0], bytes) if not ok: self._logger.error("Object value must have exactly one 'str' and one 'bytes' field") raise ValueError("Object value must have exactly one 'str' and one 'bytes' field") def _validate_payload(self, payload: list, variables: list, batch: bool): self._check_instance(payload, "Payload", list) for payload_element in payload: if batch: self._check_instance(payload_element, "Batch payload element", list) else: payload_element = [payload_element] for item in payload_element: self._validate_payload_item(item, variables) def _validate_payload_item(self, item: dict, variables: list): self._check_instance(item, "Payload item", dict) for variable in variables: name = variable["name"] value = item.get(name, None) if value is None: self._logger.warning(f"WARNING! Variable '{name}' is missing from input, output or metric") continue self._check_variable_types(variable, value) payload_names = set(item.keys()) variable_names = { variable["name"] for variable in variables } variable_names.add('timestamp') extra_variables = payload_names - variable_names if len(extra_variables): self._logger.warning(f"WARNING! These variables are not declared but are part of the payload: {extra_variables}") def _create_command(self, input_payload_path, output_payload_path): """ Create the command to run the component. """ cmd = [str(self.python_path), "-m", 'run_component'] if isinstance(self.component, PythonComponent): cmd += [ "-m", Path(self.component.entrypoint).stem, "-p", json.dumps(self.parameters) ] if isinstance(self.component, GPURuntimeComponent): model_path = Path(self.workdir / "1" / "model.onnx").absolute().resolve() config_path = Path(self.workdir / "config.pbtxt").absolute().resolve() cmd += [ "-m", str(model_path), "-c", str(config_path), ] cmd += [ "-i", input_payload_path.name, "-o", output_payload_path.name, "-ll", logging.getLevelName(self._logger.getEffectiveLevel()) ] return cmd ``` ### `set_parameters(parameter_name, parameter_value)` Set the parameters for the component. Source code in `docs/industrial-ai-suite/sdk/simaticai/testing/component_runner.py` ```python def set_parameters(self, parameter_name, parameter_value): """ Set the parameters for the component. """ self.parameters[parameter_name] = parameter_value ``` ### `run(input_payload)` Run the component with the input payload. Parameters: input_payload: Input payload for the component. Returns: Output payload from the component. Side Effects: - The input payload is saved in the workdir. - The output payload is loaded from the workdir. Source code in `docs/industrial-ai-suite/sdk/simaticai/testing/component_runner.py` ```python def run(self, input_payload: Optional[Union[dict, list]]) -> Optional[Union[dict, list]]: """ Run the component with the input payload. Parameters: input_payload: Input payload for the component. Returns: Output payload from the component. Side Effects: - The input payload is saved in the workdir. - The output payload is loaded from the workdir. """ input_payload_path: Path = self.workdir / "input.joblib" output_payload_path: Path = self.workdir / "output.joblib" batch_input: bool = self.component.batch.inputBatch if isinstance(input_payload, DataStream): input_payload = [item for item in input_payload] elif isinstance(input_payload, dict): input_payload = [input_payload] input_variables = [{'name': name, 'type': c_input['type']} for name, c_input in self.component.inputs.items()] self._validate_payload(input_payload, input_variables, batch_input) joblib.dump(input_payload, input_payload_path) cmd = self._create_command(input_payload_path, output_payload_path) self._logger.info("Running command: " + " ".join(cmd)) result = subprocess.run(cmd, cwd=self.workdir, capture_output=True) if result.returncode != 0: raise RuntimeError(result.stderr.decode()) self._logger.debug(result.stderr.decode()) output_payload = joblib.load(output_payload_path) batch_output: bool = self.component.batch.outputBatch output_payload = output_payload if isinstance(output_payload, list) else [output_payload] output_payload = [output for output in output_payload if output is not None] output_variables = [{'name': name, 'type': c_output['type']} for name, c_output in self.component.outputs.items()] if isinstance(self.component, PythonComponent): output_variables += [{'name': name, 'type': 'String'} for name in self.component.metrics] self._validate_payload(output_payload, output_variables, batch_output) return output_payload ``` ## `DataStream` Base class for datastream generators Source code in `docs/industrial-ai-suite/sdk/simaticai/testing/data_stream.py` ```python class DataStream: """ Base class for datastream generators """ def __iter__(self): """ Empty generator method for child classess to implement. """ raise NotImplementedError('Child classes must implement this method.') ``` ### `__iter__()` Empty generator method for child classess to implement. Source code in `docs/industrial-ai-suite/sdk/simaticai/testing/data_stream.py` ```python def __iter__(self): """ Empty generator method for child classess to implement. """ raise NotImplementedError('Child classes must implement this method.') ``` This module provides a base class for creating and managing Docker containers with Python virtual environments. It is designed to be used in a context where Docker is available and the user has the necessary permissions to create and manage containers. ## `VesselBaseDocker` Source code in `docs/industrial-ai-suite/sdk/simaticai/testing/docker_venv.py` ```python class VesselBaseDocker: is_vessel: bool = False # True when running in a Docker container python_3_11: str = 'python' python_3_12: str = 'python' def __init__(self): if Path("/.ai_sdk_docker").is_file(): self.is_vessel = True self.python_default = 'python3.11' self.python_3_11 = 'python3.11' self.python_3_12 = 'python3.12' def _create_venv(self, path: str, version: str): """ Creates a virtual environment in which the given component can run. Args: path (str): Path to the virtual environment. version (str): Python version to use for the virtual environment. """ context_dir = Path(path) / ".venv" python_path = context_dir / "bin" / "python" match version: case "3.11": python = self.python_3_11 case "3.12": python = self.python_3_12 case _: python = self.python_default _logger.info(f"Creating virtual environment for Python '{version}' in '{path}/.venv' with Python {python}...") command = [ python, "-m", "venv", str(context_dir), "--copies" ] result = subprocess.run(command, capture_output=True, text=True) if result.returncode != 0: _logger.error(f"Error creating virtual environment: {result.stderr}") raise RuntimeError(f"Error creating virtual environment: {result.stderr}") return context_dir.resolve(), python_path.resolve() ``` A pipeline runner that lets you simulate the execution of a pipeline in a local Python environment. It can be used to locally mimic the behavior of the AI Inference Server concerning loading and running inference pipelines. This is a quick and easy way to find programming or configuration errors before deploying the package. The local pipeline runner also lets you exercise your pipeline component by component. In other words, you can feed single components with inputs and verify the output produced. ## `LocalPipelineRunner` Simulates the execution of a pipeline in a local Python environment. Restriction: only linear pipelines are supported where the pipeline input variables are only used by one component, each component uses only the outputs of the previous components, and the pipeline output only consists of variables from the last component. If the caller specifies no `path`, the working directory is temporary and is removed unless an error occurs. If the caller specifies a working directory with a `path` argument, the working directory is kept. This behavior can be overridden using boolean parameter `cleanup`. Currently, the pipeline runner supports both the current `process_input(data: dict)` entrypoint signature and the legacy `run(data: str)` signature. If both entrypoints are present, `process_input()` takes precedence. Please note however that `run()` is deprecated, and support for it will be removed in future versions of the pipeline runner. Parameters: | Name | Type | Description | Default | | ------------ | ------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- | | `packageZip` | `path - like` | Path to the pipeline configuration package. | *required* | | `path` | `path - like` | Path to the working directory. If unset, a temporary directory is created. | `None` | | `cleanup` | `bool` | If set, the working directory is kept when True, and deleted when False. If unset, a temporary working directory is removed, and an explicit working directory is kept. When an error occurs in a component, the working directory is kept regardless of this value. | `None` | Source code in `docs/industrial-ai-suite/sdk/simaticai/testing/pipeline_runner.py` ```python class LocalPipelineRunner: """ Simulates the execution of a pipeline in a local Python environment. Restriction: only linear pipelines are supported where the pipeline input variables are only used by one component, each component uses only the outputs of the previous components, and the pipeline output only consists of variables from the last component. If the caller specifies no `path`, the working directory is temporary and is removed unless an error occurs. If the caller specifies a working directory with a `path` argument, the working directory is kept. This behavior can be overridden using boolean parameter `cleanup`. Currently, the pipeline runner supports both the current `process_input(data: dict)` entrypoint signature and the legacy `run(data: str)` signature. If both entrypoints are present, `process_input()` takes precedence. Please note however that `run()` is deprecated, and support for it will be removed in future versions of the pipeline runner. Args: packageZip (path-like): Path to the pipeline configuration package. path (path-like): Path to the working directory. If unset, a temporary directory is created. cleanup (bool): If set, the working directory is kept when True, and deleted when False. \ If unset, a temporary working directory is removed, and an explicit working directory is kept. \ When an error occurs in a component, the working directory is kept regardless of this value. """ def __init__(self, packageZip: os.PathLike, path: Optional[os.PathLike] = None, cleanup: Optional[bool] = None, loglevel = logging.INFO): """ Creates a new component LocalPipelineRunner for the provided pipeline configuration package. Only works with a pipeline configuration package. Does not work with e.g. an edge configuration package. Args: packageZip (path-like): Path to the pipeline configuration package. path (path-like): Path to the working directory. If unset a temporary directory will be created. cleanup (bool): If set, the working directory will be kept when True, and deleted when False. \ If unset, a temporary working directory will be removed, and an explicit working directory will be kept. \ When an error occurs in a component, the working directory will be kept regardless of this value. """ self.package_zip: Path = Path(packageZip) self.path = Path(path).resolve().absolute() if path is not None else None self.components = {} self.parameters = {} self.cleanup = cleanup self.log_level = loglevel self.workdir: Path self.docker = VesselBaseDocker() self.report_writer = PipelineRunnerReportWriter() report_writer_handler = ReportWriterHandler(self.report_writer) _logger.addHandler(report_writer_handler) def __enter__(self): self.report_writer.set_package_zip_path(self.package_zip) timestamp = re.sub(r"[-:]", "", datetime.now(timezone.utc).isoformat(sep="_", timespec="seconds")) if self.path is not None: self.workdir = Path(self.path) self.workdir.mkdir(parents=True, exist_ok=True) self.cleanup = self.cleanup if self.cleanup is not None else False else: self.workdir = Path(tempfile.mkdtemp(prefix=f"LocalPipelineRunner_{timestamp}_")) self.cleanup = self.cleanup if self.cleanup is not None else True unzip_components = False with zipfile.ZipFile(self.package_zip) as zf: if 'runtime_config.yml' in zf.namelist(): self.workdir = self.workdir / self.package_zip.stem zf.extractall(path=self.workdir) unzip_components = True else: zf.extractall(path=self.workdir) self.workdir = self.workdir / zf.namelist()[0] try: with open(self.workdir / "pipeline_config.yml") as cf: config = yaml.load(cf, Loader=yaml.FullLoader) components = config["dataFlowPipeline"].get("components", []) for component in components: component["context"] = None component["env_dir"] = self.workdir / component['name'] self.components[component["name"]] = component if unzip_components: for component in components: component_zip = f"{component['name']}_{component['version']}.zip" with zipfile.ZipFile(self.workdir / 'components' / component_zip) as zf: zf.extractall(path=self.workdir / component["name"]) for parameter in config["dataFlowPipeline"].get("pipelineParameters", {}): self.parameters[parameter["name"]] = parameter except Exception: raise RuntimeError(INVALID_PIPELINE_PACKAGE_MESSAGE) return self def __exit__(self, exception_type, value, traceback): self.update_telemetry_data() self.report_writer.write_report() if self.cleanup: _logger.info("Removing local pipeline runner environment...") shutil.rmtree(self.workdir.parent) else: _logger.info(f"Leaving local pipeline runner environment in its final state at '{self.workdir}'") def collect_telemetry_data(self) -> dict: """ Collects telemetry data about the platform, environment, industrial AI packages, and pipeline. Returns: dict: A dictionary containing the telemetry data. """ telemetry_data = {} telemetry_data["platform"] = {} telemetry_data["platform"]["os"] = platform.system() telemetry_data["platform"]["release"] = platform.release() telemetry_data["platform"]["python_version"] = platform.python_version() _logger.info(f"locals: {locals()}") telemetry_data["environment"] = {} env_info = "Jupyter" if any(k for k in locals() if k in ["__IPYTHON__", "get_ipython"]) else MSG_NOT_FOUND env_info = "GitLab CI/CD" if "GITLAB_CI" in os.environ else env_info env_info = "Azure DevOps Pipelines" if "TF_BUILD" in os.environ else env_info env_info = "GitHub Actions" if "GITHUB_ACTIONS" in os.environ else env_info telemetry_data["environment"]["info"] = env_info telemetry_data["industrial_ai"] = {} try: telemetry_data["industrial_ai"]["simaticai"] = importlib_metadata.version("simaticai") except importlib_metadata.PackageNotFoundError: telemetry_data["industrial_ai"]["simaticai"] = MSG_NOT_FOUND telemetry_data["pipeline"] = {} telemetry_data["pipeline"]["python_versions"] = list(set(self.components[component]['runtime']['version'] for component in self.components if self.components[component]['runtime']['type'] == 'python')) telemetry_data["pipeline"]["file_extensions"] = [] for component_dir in [Path(self.workdir) / c for c in Path(self.workdir).rglob("*") if c.name in self.components.keys()]: excluded_dirs = set([component_dir / '.venv', component_dir / '__pyache__']) suffixes = list(set(f.suffix for f in component_dir.rglob("*") if not (any(excluded_dirs.intersection(f.parents)) or f.suffix in ["", ".zip", ".yml", ".yaml", ".html"]))) for suffix in suffixes: if suffix not in telemetry_data["pipeline"]["file_extensions"]: telemetry_data["pipeline"]["file_extensions"].append(suffix) return telemetry_data def update_telemetry_data(self): """ Update the telemetry data and the package. This method updates the telemetry data by loading the existing data from a YAML file, or collecting new telemetry data if the file doesn't exist. It then updates the "last_test_run" field of the telemetry data with the current timestamp. The updated telemetry data is then written back to the YAML file. If the package contains a different version of the telemetry data file, a new package is created with the updated telemetry data. Otherwise, the existing package is overwritten with the new package containing the updated telemetry data. """ _logger.info("Updating telemetry data and the package") telemetry_path = self.workdir / "telemetry_data.yml" if telemetry_path.is_file(): telemetry_data = yaml.safe_load(telemetry_path.read_text()) else: telemetry_data = self.collect_telemetry_data() telemetry_data["pipeline"]["last_test_run"] = datetime.now().isoformat() telemetry_path.write_text(yaml.dump(telemetry_data)) config_package = False with zipfile.ZipFile(self.package_zip, 'r') as zip_read: for file in zip_read.namelist(): if TELEMETRY_YAML in file and file != TELEMETRY_YAML: config_package = True break new_zip_path = Path(self.package_zip).parent / f"{Path(self.package_zip).stem}_tested.zip" with zipfile.ZipFile(self.package_zip, 'r') as zip_read: with zipfile.ZipFile(new_zip_path, 'w') as zip_write: for file in zip_read.namelist(): if TELEMETRY_YAML not in file: filepath = zip_read.extract(file, path=self.workdir) zip_write.write(filepath, arcname=file) else: zip_write.write(telemetry_path, arcname=file) if config_package: shutil.copy(new_zip_path, self.package_zip) else: new_sha_path = Path(self.package_zip).parent / f"{Path(self.package_zip).stem}_tested.sha256" new_sha_path.write_text(calc_sha(new_zip_path)) def _install_requirements(self, component, package_dir, no_index: bool = True): _logger.info("Installing requirements...") pip_report_file = component["env_dir"] / "pip_report.json" package_dir_path = _relative_to(package_dir, component["env_dir"]) pip_report_file_path = _relative_to(pip_report_file, component["env_dir"]) cmd = [ str(component['python_path']), "-m", "pip", "install", "--no-warn-script-location", "-f", f"{package_dir_path}", "-r", REQUIREMENTS_TXT, ] if no_index: cmd += [ "--no-index" ] use_no_deps = component.get("runtime", {}).get("installTransitiveDependencies", True) is False if use_no_deps: cmd += [ "--no-deps" ] result = subprocess.run(cmd, cwd=component["env_dir"], text=True, stderr=subprocess.PIPE) if 0 == result.returncode and use_no_deps: _logger.warning(f"Component `{component['name']}` is requested to be installed without transitive dependencies. " "This may lead to runtime errors if the dependencies are not handled manually.") # generate pip report with a dry run to get the list of installed packages (in case of CPU Python packages) if 0 == result.returncode and component.get("hwType", None) == "CPU": cmd += ["--dry-run", "--ignore-installed", "--report", f"{pip_report_file_path}"] subprocess.run(cmd, cwd=component["env_dir"], text=True, stderr=subprocess.PIPE) self.report_writer.add_installed_packages(component["name"], pip_report_file) return result def _install_from_packages_zip(self, component, package_dir): result = self._install_requirements(component, package_dir, True) return 0 == result.returncode def _install_from_pypi_org(self, component, package_dir): return self._install_requirements(component, package_dir, False) def _init_component_venv(self, component: dict): """ Creates a virtual environment in which the given component can run. Args: component (str): name of the selected component. """ _logger.info(f"Creating virtual environment for component '{component['name']}'...") context_dir = component["env_dir"] / ".venv" if self.docker.is_vessel: _logger.info("Creating virtual environment in docker...") component["context"], component["python_path"] = self.docker._create_venv(component["env_dir"], component["runtime"]["version"]) component["python_path"] = Path(component["python_path"]).resolve() _logger.info(f"Component context (docker): {component['context']}") else: builder = venv.EnvBuilder(with_pip=True, symlinks=False) builder.create(str(context_dir)) component["context"] = builder.ensure_directories(context_dir) component['python_path'] = Path(component["context"].env_exe).resolve() _logger.debug(f"Component python_path: {component['python_path']}") _logger.info("Upgrading pip...") cmd = [str(component['python_path']), "-m", "pip", "install", "pip", "--upgrade"] result = subprocess.run(cmd, cwd=component["env_dir"], text=True, stderr=subprocess.PIPE) if result.returncode != 0: self.cleanup = False _logger.warning(f"Error upgrading pip:\n{result.stderr}") cmd = [str(component['python_path']), "-m", "pip", "install", "--upgrade", "setuptools"] result = subprocess.run(cmd, cwd=component["env_dir"], text=True, stderr=subprocess.PIPE) if result.returncode != 0: self.cleanup = False _logger.warning(f"Error installing setuptools:\n{result.stderr}") try: result = self._install_logmodule(component["python_path"], component["env_dir"]) except Exception as err: _logger.error(err) self.cleanup = False raise RuntimeError("The 'simaticai' Python package is either not installed or does not contain package 'log_module'.") from None if result.returncode != 0: self.cleanup = False raise RuntimeError(f"Error installing log_module:\n{result.stderr}") req_list = Path(component["env_dir"] / "requirements.list") req_list.touch(exist_ok=True) if Path(component["env_dir"] / REQUIREMENTS_TXT).is_file(): dependencies, extra_index, index_url = parse_requirements(component["env_dir"] / REQUIREMENTS_TXT) requirements = "#".join(dependencies.keys()) req_list.write_text(requirements) else: _logger.info(f"'{REQUIREMENTS_TXT}' was not found. No additional dependencies were installed.") return package_dir = component["env_dir"] / PYTHON_PACKAGES package_zip = component["env_dir"] / PYTHON_PACKAGES_ZIP _logger.info(f"Extracting {PYTHON_PACKAGES_ZIP}") if package_zip.is_file(): with zipfile.ZipFile(package_zip) as zf: zf.extractall(path=package_dir.absolute()) else: _logger.info(f"There is no {PYTHON_PACKAGES_ZIP} to extract.") package_dir.mkdir(parents=True, exist_ok=True) success = self._install_from_packages_zip(component, package_dir) if not success: msg = f"Warning! Could not install dependencies from {PYTHON_PACKAGES_ZIP}. " msg += "Trying to install them from pypi.org. The resulting Python environment " msg += "may be significantly different than the targeted Python environment on the Edge Device!" _logger.warning(msg) if self.docker.is_vessel and component["runtime"]["type"] == "python": raise RuntimeError("The component is running in a docker container. The installation of dependencies from pypi.org is not supported.") else: second_install_result = self._install_from_pypi_org(component, package_dir) if 0 != second_install_result.returncode: self.cleanup = False raise RuntimeError(f"Error installing requirements:\n{second_install_result.stderr}") @staticmethod def _install_logmodule(python_path, env_dir): _logger.info("Installing LogModule...") try: package_paths = importlib_metadata.files("simaticai") assert package_paths is not None logger_wheel = [p for p in package_paths if 'log_module' in str(p)][0].locate() except Exception: from importlib.metadata import Distribution direct_url = Distribution.from_name("simaticai").read_text("direct_url.json") assert direct_url is not None direct_url = json.loads(direct_url)['url'] direct_url = direct_url.replace('file://','') direct_url = Path(direct_url) / 'simaticai' / 'data' paths = list(direct_url.rglob('*.whl')) logger_wheel = [p for p in paths if 'log_module' in str(p)][0].resolve() cmd = [ str(python_path), "-m", "pip", "install", "--no-warn-script-location", logger_wheel, "joblib" ] return subprocess.run(cmd, cwd=env_dir, text=True, stderr=subprocess.PIPE) def run_component(self, name: str, data: Optional[Union[dict, list, DataStream]]) -> Optional[Union[dict, list]]: """ Runs the component in its virtual environment with the given input. This environment is created according to `requirements.txt` in the package. Additionally 'joblib' and the mock `log_module` is automatically installed in this virtual environment. The input data can be a single input record in a dictionary or a batch of input records in a list of dictionaries, or a DataStream object which will produce the appropriate input data. The supplied input data is saved as `inputs.joblib` in the component runtime directory, and the output is saved as `output.joblib`. Args: name (str): The name of the component to be executed. data (dict or list): One or more input records for the component. Returns: dict / list: A list of dictionaries for outputs if there were no errors and field `ready` is true. If the input was a single dict, then a single dict (the first item of the list) or None if there is no output. """ assert name in self.components, f"Invalid component name: {name}" component = self.components[name] assert component["runtime"]["type"] in ["python", "gpuruntime"], f"Can not run component '{name}': Runtime type is nor 'python' or 'gpuruntime'" input_payload_path: Path = component["env_dir"] / "input.joblib" output_payload_path: Path = component["env_dir"] / "output.joblib" batch_input: bool = component["batch"]["inputBatch"] == "Yes" if component.get("batch") is not None else False batch_output: bool = component["batch"]["outputBatch"] == "Yes" if component.get("batch") is not None else False # Validate and serialize input payload assert data is not None, f"Can not run component '{name}' without input." result_is_list = True if isinstance(data, list): input_payload = data elif isinstance(data, DataStream): input_payload = [item for item in data] else: result_is_list = False input_payload = [data] validate_payload(input_payload, component["inputType"], batch_input) joblib.dump(input_payload, input_payload_path) self.report_writer.set_input_payload_length(name, len(input_payload)) _logger.info(f"Input payload saved as '{input_payload_path}'") # Assemble command for runnig component if component['runtime']['type'] == 'python': # Version check for Python e_major, e_minor, _, _, _ = sys.version_info c_major, c_minor, *_ = tuple(str(component["runtime"]["version"]).split('.')) if not self.docker.is_vessel and (f"{e_major}.{e_minor}" != f"{c_major}.{c_minor}"): msg = f"The local python version ({e_major}.{e_minor}) and the python version defined for the component ({c_major}.{c_minor}) are different." msg += " Testing will be done using dependencies that corresponds to the python version of your development environment." msg += " Pipeline behavior on AI Inference Server might be different." _logger.warning(msg) if component["context"] is None: self._init_component_venv(component) shutil.copy(_runner_path / 'run_component.py', component["env_dir"]) req_list_path = _relative_to(component["env_dir"] / "requirements.list", component["env_dir"]) json_params = json.dumps({param["name"]: param["defaultValue"] for param in self.parameters.values()}) args = [ "-m", 'run_component', "-m", Path(component["entrypoint"]).stem, "-p", f"{json_params}", "-r", f"{req_list_path}", ] else: # gpuruntime step requires Python environment with onnxruntime installed if component["context"] is None: shutil.copy(_runner_path / 'gpuruntime_requirements.txt', component["env_dir"] / REQUIREMENTS_TXT) self._init_component_venv(component) shutil.copy(_runner_path / 'run_gpuruntime_component.py', component["env_dir"]) shutil.copy(_runner_path.parent / 'model_config_pb2.py', component["env_dir"]) args = [ "-m", 'run_gpuruntime_component', "-m", component["entrypoint"], "-c", "config.pbtxt", ] args += [ "-i", input_payload_path.name, "-o", output_payload_path.name, "-ll", logging.getLevelName(self.log_level) ] # Run the component in the created Python environment _logger.info(f"Running component '{name}'...") _logger.info(f"{component['python_path'] =} to {component['env_dir'] =}") cmd = [str(component['python_path'])] + args _logger.info(f"Running command: {subprocess.list2cmdline(cmd)}") p = subprocess.Popen(cmd, cwd=component["env_dir"], stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True) if p.stdout is not None: for line in p.stdout: _logger.info(line.strip()) returncode = p.wait() if returncode not in [RETURN_CODE_OK, RETURN_CODE_DEPRECATED]: self.cleanup = False raise RuntimeError(f"""\ There was an error while running component '{name}'. You can check the test results in directory '{component['env_dir']}' """) else: response_wrapped = True if returncode == RETURN_CODE_DEPRECATED else False # Deserialize and validate output payload _logger.info(f"Loading output payload from '{output_payload_path}'") output_payload = joblib.load(output_payload_path) output_payload = output_payload if isinstance(output_payload, list) else [output_payload] self.report_writer.set_output_payload_length(name, len(output_payload)) if response_wrapped: output_payload = [json.loads(item['output']) for item in output_payload if item['ready']] else: output_payload = [output for output in output_payload if output is not None] validate_payload(output_payload, component["outputType"], batch_output) if result_is_list: return output_payload if 0 < len(output_payload): return output_payload[0] return None def update_parameters(self, parameters: dict): """ Validates and updates pipeline parameters. The elements of the dictionary must match the parameters specified in the pipeline configuration package. If any of the names or types does not match, all parameters will remain untouched. Args: parameters (dict): names and values of parameters to update Raises: AssertionError: When: - either `name` is not in the configured parameters, - or `defaultValue` type is different from the configured one """ for key,value in parameters.items(): if key not in self.parameters.keys(): if key.upper().startswith("__AI_IS"): self.parameters[key] = {'defaultValue': False, 'name': key, 'type': 'Boolean'} else: raise AssertionError(f"Pipeline has no parameters with the name '{key}' and type '{type_map.get(type(value).__name__)}'") elif self.parameters[key]["type"] != type_map.get(type(value).__name__): raise AssertionError(f"Pipeline has no parameters with the name '{key}' and type '{type_map.get(type(value).__name__)}'") for key, value in parameters.items(): self.parameters[key]["defaultValue"] = value def run_pipeline(self, payload: Optional[Union[dict, list, DataStream]] = {}) -> Optional[Union[dict, list]]: """ Runs all the components sequentially, assuming the output of a component is only consumed by the next. The input data can be a single input record in a dictionary or a batch of input records in a list of dictionaries. For each component the supplied input data is saved as `input.joblib` in the component runtime directory, and the output is saved as `output.joblib`. Args: payload (dict or list): One or more input records for the pipeline. Returns: The output of the last component. """ for name in self.components.keys(): payload = self.run_component(name, payload) return payload ``` ### `__init__(packageZip, path=None, cleanup=None, loglevel=logging.INFO)` Creates a new component LocalPipelineRunner for the provided pipeline configuration package. Only works with a pipeline configuration package. Does not work with e.g. an edge configuration package. Parameters: | Name | Type | Description | Default | | ------------ | ------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- | | `packageZip` | `path - like` | Path to the pipeline configuration package. | *required* | | `path` | `path - like` | Path to the working directory. If unset a temporary directory will be created. | `None` | | `cleanup` | `bool` | If set, the working directory will be kept when True, and deleted when False. If unset, a temporary working directory will be removed, and an explicit working directory will be kept. When an error occurs in a component, the working directory will be kept regardless of this value. | `None` | Source code in `docs/industrial-ai-suite/sdk/simaticai/testing/pipeline_runner.py` ```python def __init__(self, packageZip: os.PathLike, path: Optional[os.PathLike] = None, cleanup: Optional[bool] = None, loglevel = logging.INFO): """ Creates a new component LocalPipelineRunner for the provided pipeline configuration package. Only works with a pipeline configuration package. Does not work with e.g. an edge configuration package. Args: packageZip (path-like): Path to the pipeline configuration package. path (path-like): Path to the working directory. If unset a temporary directory will be created. cleanup (bool): If set, the working directory will be kept when True, and deleted when False. \ If unset, a temporary working directory will be removed, and an explicit working directory will be kept. \ When an error occurs in a component, the working directory will be kept regardless of this value. """ self.package_zip: Path = Path(packageZip) self.path = Path(path).resolve().absolute() if path is not None else None self.components = {} self.parameters = {} self.cleanup = cleanup self.log_level = loglevel self.workdir: Path self.docker = VesselBaseDocker() self.report_writer = PipelineRunnerReportWriter() report_writer_handler = ReportWriterHandler(self.report_writer) _logger.addHandler(report_writer_handler) ``` ### `collect_telemetry_data()` Collects telemetry data about the platform, environment, industrial AI packages, and pipeline. Returns: | Name | Type | Description | | ------ | ------ | ------------------------------------------- | | `dict` | `dict` | A dictionary containing the telemetry data. | Source code in `docs/industrial-ai-suite/sdk/simaticai/testing/pipeline_runner.py` ```python def collect_telemetry_data(self) -> dict: """ Collects telemetry data about the platform, environment, industrial AI packages, and pipeline. Returns: dict: A dictionary containing the telemetry data. """ telemetry_data = {} telemetry_data["platform"] = {} telemetry_data["platform"]["os"] = platform.system() telemetry_data["platform"]["release"] = platform.release() telemetry_data["platform"]["python_version"] = platform.python_version() _logger.info(f"locals: {locals()}") telemetry_data["environment"] = {} env_info = "Jupyter" if any(k for k in locals() if k in ["__IPYTHON__", "get_ipython"]) else MSG_NOT_FOUND env_info = "GitLab CI/CD" if "GITLAB_CI" in os.environ else env_info env_info = "Azure DevOps Pipelines" if "TF_BUILD" in os.environ else env_info env_info = "GitHub Actions" if "GITHUB_ACTIONS" in os.environ else env_info telemetry_data["environment"]["info"] = env_info telemetry_data["industrial_ai"] = {} try: telemetry_data["industrial_ai"]["simaticai"] = importlib_metadata.version("simaticai") except importlib_metadata.PackageNotFoundError: telemetry_data["industrial_ai"]["simaticai"] = MSG_NOT_FOUND telemetry_data["pipeline"] = {} telemetry_data["pipeline"]["python_versions"] = list(set(self.components[component]['runtime']['version'] for component in self.components if self.components[component]['runtime']['type'] == 'python')) telemetry_data["pipeline"]["file_extensions"] = [] for component_dir in [Path(self.workdir) / c for c in Path(self.workdir).rglob("*") if c.name in self.components.keys()]: excluded_dirs = set([component_dir / '.venv', component_dir / '__pyache__']) suffixes = list(set(f.suffix for f in component_dir.rglob("*") if not (any(excluded_dirs.intersection(f.parents)) or f.suffix in ["", ".zip", ".yml", ".yaml", ".html"]))) for suffix in suffixes: if suffix not in telemetry_data["pipeline"]["file_extensions"]: telemetry_data["pipeline"]["file_extensions"].append(suffix) return telemetry_data ``` ### `update_telemetry_data()` Update the telemetry data and the package. This method updates the telemetry data by loading the existing data from a YAML file, or collecting new telemetry data if the file doesn't exist. It then updates the "last_test_run" field of the telemetry data with the current timestamp. The updated telemetry data is then written back to the YAML file. If the package contains a different version of the telemetry data file, a new package is created with the updated telemetry data. Otherwise, the existing package is overwritten with the new package containing the updated telemetry data. Source code in `docs/industrial-ai-suite/sdk/simaticai/testing/pipeline_runner.py` ```python def update_telemetry_data(self): """ Update the telemetry data and the package. This method updates the telemetry data by loading the existing data from a YAML file, or collecting new telemetry data if the file doesn't exist. It then updates the "last_test_run" field of the telemetry data with the current timestamp. The updated telemetry data is then written back to the YAML file. If the package contains a different version of the telemetry data file, a new package is created with the updated telemetry data. Otherwise, the existing package is overwritten with the new package containing the updated telemetry data. """ _logger.info("Updating telemetry data and the package") telemetry_path = self.workdir / "telemetry_data.yml" if telemetry_path.is_file(): telemetry_data = yaml.safe_load(telemetry_path.read_text()) else: telemetry_data = self.collect_telemetry_data() telemetry_data["pipeline"]["last_test_run"] = datetime.now().isoformat() telemetry_path.write_text(yaml.dump(telemetry_data)) config_package = False with zipfile.ZipFile(self.package_zip, 'r') as zip_read: for file in zip_read.namelist(): if TELEMETRY_YAML in file and file != TELEMETRY_YAML: config_package = True break new_zip_path = Path(self.package_zip).parent / f"{Path(self.package_zip).stem}_tested.zip" with zipfile.ZipFile(self.package_zip, 'r') as zip_read: with zipfile.ZipFile(new_zip_path, 'w') as zip_write: for file in zip_read.namelist(): if TELEMETRY_YAML not in file: filepath = zip_read.extract(file, path=self.workdir) zip_write.write(filepath, arcname=file) else: zip_write.write(telemetry_path, arcname=file) if config_package: shutil.copy(new_zip_path, self.package_zip) else: new_sha_path = Path(self.package_zip).parent / f"{Path(self.package_zip).stem}_tested.sha256" new_sha_path.write_text(calc_sha(new_zip_path)) ``` ### `run_component(name, data)` Runs the component in its virtual environment with the given input. This environment is created according to `requirements.txt` in the package. Additionally 'joblib' and the mock `log_module` is automatically installed in this virtual environment. The input data can be a single input record in a dictionary or a batch of input records in a list of dictionaries, or a DataStream object which will produce the appropriate input data. The supplied input data is saved as `inputs.joblib` in the component runtime directory, and the output is saved as `output.joblib`. Parameters: | Name | Type | Description | Default | | ------ | -------------- | -------------------------------------------- | ---------- | | `name` | `str` | The name of the component to be executed. | *required* | | `data` | `dict or list` | One or more input records for the component. | *required* | Returns: | Type | Description | | ----------------------------- | -------------------------------------------------------------------------------------------------------------- | | `Optional[Union[dict, list]]` | dict / list: A list of dictionaries for outputs if there were no errors and field ready is true. | | `Optional[Union[dict, list]]` | If the input was a single dict, then a single dict (the first item of the list) or None if there is no output. | Source code in `docs/industrial-ai-suite/sdk/simaticai/testing/pipeline_runner.py` ```python def run_component(self, name: str, data: Optional[Union[dict, list, DataStream]]) -> Optional[Union[dict, list]]: """ Runs the component in its virtual environment with the given input. This environment is created according to `requirements.txt` in the package. Additionally 'joblib' and the mock `log_module` is automatically installed in this virtual environment. The input data can be a single input record in a dictionary or a batch of input records in a list of dictionaries, or a DataStream object which will produce the appropriate input data. The supplied input data is saved as `inputs.joblib` in the component runtime directory, and the output is saved as `output.joblib`. Args: name (str): The name of the component to be executed. data (dict or list): One or more input records for the component. Returns: dict / list: A list of dictionaries for outputs if there were no errors and field `ready` is true. If the input was a single dict, then a single dict (the first item of the list) or None if there is no output. """ assert name in self.components, f"Invalid component name: {name}" component = self.components[name] assert component["runtime"]["type"] in ["python", "gpuruntime"], f"Can not run component '{name}': Runtime type is nor 'python' or 'gpuruntime'" input_payload_path: Path = component["env_dir"] / "input.joblib" output_payload_path: Path = component["env_dir"] / "output.joblib" batch_input: bool = component["batch"]["inputBatch"] == "Yes" if component.get("batch") is not None else False batch_output: bool = component["batch"]["outputBatch"] == "Yes" if component.get("batch") is not None else False # Validate and serialize input payload assert data is not None, f"Can not run component '{name}' without input." result_is_list = True if isinstance(data, list): input_payload = data elif isinstance(data, DataStream): input_payload = [item for item in data] else: result_is_list = False input_payload = [data] validate_payload(input_payload, component["inputType"], batch_input) joblib.dump(input_payload, input_payload_path) self.report_writer.set_input_payload_length(name, len(input_payload)) _logger.info(f"Input payload saved as '{input_payload_path}'") # Assemble command for runnig component if component['runtime']['type'] == 'python': # Version check for Python e_major, e_minor, _, _, _ = sys.version_info c_major, c_minor, *_ = tuple(str(component["runtime"]["version"]).split('.')) if not self.docker.is_vessel and (f"{e_major}.{e_minor}" != f"{c_major}.{c_minor}"): msg = f"The local python version ({e_major}.{e_minor}) and the python version defined for the component ({c_major}.{c_minor}) are different." msg += " Testing will be done using dependencies that corresponds to the python version of your development environment." msg += " Pipeline behavior on AI Inference Server might be different." _logger.warning(msg) if component["context"] is None: self._init_component_venv(component) shutil.copy(_runner_path / 'run_component.py', component["env_dir"]) req_list_path = _relative_to(component["env_dir"] / "requirements.list", component["env_dir"]) json_params = json.dumps({param["name"]: param["defaultValue"] for param in self.parameters.values()}) args = [ "-m", 'run_component', "-m", Path(component["entrypoint"]).stem, "-p", f"{json_params}", "-r", f"{req_list_path}", ] else: # gpuruntime step requires Python environment with onnxruntime installed if component["context"] is None: shutil.copy(_runner_path / 'gpuruntime_requirements.txt', component["env_dir"] / REQUIREMENTS_TXT) self._init_component_venv(component) shutil.copy(_runner_path / 'run_gpuruntime_component.py', component["env_dir"]) shutil.copy(_runner_path.parent / 'model_config_pb2.py', component["env_dir"]) args = [ "-m", 'run_gpuruntime_component', "-m", component["entrypoint"], "-c", "config.pbtxt", ] args += [ "-i", input_payload_path.name, "-o", output_payload_path.name, "-ll", logging.getLevelName(self.log_level) ] # Run the component in the created Python environment _logger.info(f"Running component '{name}'...") _logger.info(f"{component['python_path'] =} to {component['env_dir'] =}") cmd = [str(component['python_path'])] + args _logger.info(f"Running command: {subprocess.list2cmdline(cmd)}") p = subprocess.Popen(cmd, cwd=component["env_dir"], stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True) if p.stdout is not None: for line in p.stdout: _logger.info(line.strip()) returncode = p.wait() if returncode not in [RETURN_CODE_OK, RETURN_CODE_DEPRECATED]: self.cleanup = False raise RuntimeError(f"""\ There was an error while running component '{name}'. You can check the test results in directory '{component['env_dir']}' """) else: response_wrapped = True if returncode == RETURN_CODE_DEPRECATED else False # Deserialize and validate output payload _logger.info(f"Loading output payload from '{output_payload_path}'") output_payload = joblib.load(output_payload_path) output_payload = output_payload if isinstance(output_payload, list) else [output_payload] self.report_writer.set_output_payload_length(name, len(output_payload)) if response_wrapped: output_payload = [json.loads(item['output']) for item in output_payload if item['ready']] else: output_payload = [output for output in output_payload if output is not None] validate_payload(output_payload, component["outputType"], batch_output) if result_is_list: return output_payload if 0 < len(output_payload): return output_payload[0] return None ``` ### `update_parameters(parameters)` Validates and updates pipeline parameters. The elements of the dictionary must match the parameters specified in the pipeline configuration package. If any of the names or types does not match, all parameters will remain untouched. Args: ```text parameters (dict): names and values of parameters to update ``` Raises: ```text AssertionError: When: - either `name` is not in the configured parameters, - or `defaultValue` type is different from the configured one ``` Source code in `docs/industrial-ai-suite/sdk/simaticai/testing/pipeline_runner.py` ```python def update_parameters(self, parameters: dict): """ Validates and updates pipeline parameters. The elements of the dictionary must match the parameters specified in the pipeline configuration package. If any of the names or types does not match, all parameters will remain untouched. Args: parameters (dict): names and values of parameters to update Raises: AssertionError: When: - either `name` is not in the configured parameters, - or `defaultValue` type is different from the configured one """ for key,value in parameters.items(): if key not in self.parameters.keys(): if key.upper().startswith("__AI_IS"): self.parameters[key] = {'defaultValue': False, 'name': key, 'type': 'Boolean'} else: raise AssertionError(f"Pipeline has no parameters with the name '{key}' and type '{type_map.get(type(value).__name__)}'") elif self.parameters[key]["type"] != type_map.get(type(value).__name__): raise AssertionError(f"Pipeline has no parameters with the name '{key}' and type '{type_map.get(type(value).__name__)}'") for key, value in parameters.items(): self.parameters[key]["defaultValue"] = value ``` ### `run_pipeline(payload={})` Runs all the components sequentially, assuming the output of a component is only consumed by the next. The input data can be a single input record in a dictionary or a batch of input records in a list of dictionaries. For each component the supplied input data is saved as `input.joblib` in the component runtime directory, and the output is saved as `output.joblib`. Parameters: | Name | Type | Description | Default | | --------- | -------------- | ------------------------------------------- | ------- | | `payload` | `dict or list` | One or more input records for the pipeline. | `{}` | Returns: | Type | Description | | ----------------------------- | --------------------------------- | | `Optional[Union[dict, list]]` | The output of the last component. | Source code in `docs/industrial-ai-suite/sdk/simaticai/testing/pipeline_runner.py` ```python def run_pipeline(self, payload: Optional[Union[dict, list, DataStream]] = {}) -> Optional[Union[dict, list]]: """ Runs all the components sequentially, assuming the output of a component is only consumed by the next. The input data can be a single input record in a dictionary or a batch of input records in a list of dictionaries. For each component the supplied input data is saved as `input.joblib` in the component runtime directory, and the output is saved as `output.joblib`. Args: payload (dict or list): One or more input records for the pipeline. Returns: The output of the last component. """ for name in self.components.keys(): payload = self.run_component(name, payload) return payload ``` ## `validate_payload(data, variables, batch, logger=_logger)` Validates that data is a valid list of input or output payload items. Variables list what variables each playload item has. Batch indicates if the payload items are themselves batches of items or not. Source code in `docs/industrial-ai-suite/sdk/simaticai/testing/pipeline_runner.py` ```python def validate_payload(data: list, variables: list, batch: bool, logger = _logger): """ Validates that data is a valid list of input or output payload items. Variables list what variables each playload item has. Batch indicates if the payload items are themselves batches of items or not. """ global extra_variables_to_log extra_variables_to_log = set() assert isinstance(data, list), "payload data must be a 'list'" for i in data: if batch: assert isinstance(i, list), "batch payload items must be 'list' instances" else: i = [i] for j in i: validate_payload_item(j, variables, logger) if len(extra_variables_to_log): logger.warning(f"WARNING! These variables are not declared but are part of the payload: {extra_variables_to_log}") ``` ## `validate_payload_item(data, variables, logger)` Validates that data is a valid payload item. Variables listed must have a corresponding field in data. The types of the values must match their declared type. Source code in `docs/industrial-ai-suite/sdk/simaticai/testing/pipeline_runner.py` ```python def validate_payload_item(data: dict, variables: list, logger): """ Validates that data is a valid payload item. Variables listed must have a corresponding field in data. The types of the values must match their declared type. """ assert isinstance(data, dict), "payload items must be 'dict' isntances" for variable in variables: name = variable["name"] value = data.get(name, None) if value is None: logger.warning(f"WARNING! Variable '{name}' is missing from input, output or metric") continue if variable["type"] == "String": assert isinstance(value, str), "'String' value must be an 'str'" if variable["type"] == "StringArray": assert isinstance(value, list), "'StringArray' value must be a 'list'" assert all(isinstance(i, str) for i in value), "'StringArray' items must be 'str' isntances" if variable["type"] == "Object": assert isinstance(value, dict), "'Object' value must be a 'dict'" values = list(value.values()) assert len(values) == 2, "'Object' value must have exactly 2 items" ok = isinstance(values[0], str) and isinstance(values[1], bytes) or isinstance(values[1], str) and isinstance(values[0], bytes) assert ok, "'Object' value must have exactly one 'str' and one 'bytes' field" payload_names = set(data.keys()) variable_names = { variable["name"] for variable in variables } variable_names.add('timestamp') extra_variables = payload_names - variable_names extra_variables_to_log.update(extra_variables) ``` Static validation of pipeline packages. Executes static checks on a pipeline configuration package including: - Verifying that the Python version required in the package is supported by a known version of the AI Inference Server. - Verifying that all the required Python packages are either included in the pipeline package itself or available on `pypi.org` for the target platform. ## `PipelineValidationError` Bases: `Exception` Represents a problem with the pipeline configuration. Parameters: | Name | Type | Description | Default | | ------------- | ----- | --------------------------------------------- | ---------- | | `description` | `str` | Description of the error. Mandatory argument. | *required* | Source code in `docs/industrial-ai-suite/sdk/simaticai/testing/pipeline_validator.py` ```python class PipelineValidationError(Exception): """ Represents a problem with the pipeline configuration. Args: description (str): Description of the error. Mandatory argument. """ def __init__(self, description: str) -> None: assert description is not None self.value = description def __str__(self): return self.value ``` ## `validate_pipeline_dependencies(zip_path)` @Deprecated, reason: In the future only the edge package will be generated and the same validation is performed during edge package creation. Validates an already built pipeline configuration package to check if it is compatible with the AI Inference Server. This method verifies that the requirements identified by name and version are either included in `PythonPackages.zip` or available on pypi.org for the target platform. If the required dependency for the target platform is not available on pypi.org and not present in `PythonPackages.zip` it will log the problem at the ERROR level. Parameters: | Name | Type | Description | Default | | ---------- | ------------- | --------------------------------------------------- | ---------- | | `zip_path` | `path - like` | Path to the pipeline configuration package zip file | *required* | Raises: | Type | Description | | ------------------------- | ----------------------------------------------------------- | | `PipelineValidationError` | if the validation fails. See the logger output for details. | Source code in `docs/industrial-ai-suite/sdk/simaticai/testing/pipeline_validator.py` ```python def validate_pipeline_dependencies(zip_path): """ @Deprecated, reason: In the future only the edge package will be generated and the same validation is performed during edge package creation. Validates an already built pipeline configuration package to check if it is compatible with the AI Inference Server. This method verifies that the requirements identified by name and version are either included in `PythonPackages.zip` or available on pypi.org for the target platform. If the required dependency for the target platform is not available on pypi.org and not present in `PythonPackages.zip` it will log the problem at the ERROR level. Args: zip_path (path-like): Path to the pipeline configuration package zip file Raises: PipelineValidationError: if the validation fails. See the logger output for details. """ with tempfiles.OpenZipInTemp(zip_path) as package_dir: package_dir = next(package_dir.iterdir()) error = read_config_and_download_deps(package_dir) if error: raise PipelineValidationError("Requirements of one or more components can not be satisfied.") _logger.info(f"Validating pipeline package '{zip_path}' was successful.") ``` ## `download_component_dependencies(component, package_dir)` Download the dependencies of a pipeline component. Parameters: | Name | Type | Description | Default | | ------------- | ------ | -------------------------------------------------------------- | ---------- | | `component` | `dict` | A single component from the parsed pipeline_configuration.yml. | *required* | | `package_dir` | `Path` | The directory where the component was extracted | *required* | Source code in `docs/industrial-ai-suite/sdk/simaticai/testing/pipeline_validator.py` ```python def download_component_dependencies(component: dict, package_dir: Path): """ Download the dependencies of a pipeline component. Args: component (dict): A single component from the parsed `pipeline_configuration.yml`. package_dir (Path): The directory where the component was extracted """ _logger.info(f"Validating requirements of component: {component['name']}") try: deployment.python_version_validator(component['runtime']['version']) except ValueError as error: raise PipelineValidationError(error) component_dir = package_dir / component['name'] requirements_file_path = component_dir / REQUIREMENTS_TXT python_packages_folder = _build_python_packages_folder(component_dir) return not _are_dependencies_available(requirements_file_path, component['runtime']['version'], python_packages_folder) ``` ## `read_config_and_download_deps(package_dir)` Reads the pipeline configuration from the package directory and downloads its dependencies Parameters: | Name | Type | Description | Default | | ------------- | ------ | --------------------------------------------------------------------- | ---------- | | `package_dir` | `Path` | The directory where the pipeline configuration package was extracted. | *required* | Returns: | Name | Type | Description | | ------ | ------ | --------------------------------------------------------------------------------- | | `bool` | `bool` | True if there was an error during the download of the components, False otherwise | Source code in `docs/industrial-ai-suite/sdk/simaticai/testing/pipeline_validator.py` ```python def read_config_and_download_deps(package_dir: Path) -> bool: """ Reads the pipeline configuration from the package directory and downloads its dependencies Args: package_dir (Path): The directory where the pipeline configuration package was extracted. Returns: bool: True if there was an error during the download of the components, False otherwise """ try: config = yaml_helper.read_yaml(package_dir / 'pipeline_config.yml') error = False for component in config['dataFlowPipeline']['components']: error = download_component_dependencies(component, package_dir) or error return error except Exception: raise PipelineValidationError(INVALID_PIPELINE_PACKAGE_MESSAGE) ``` Utility script for running an `entrypoint` Python script in a given virtual Python environment. It is designed to be executed from `simaticai.testing.PipelineRunner` class. It consumes input data from a joblib file and produces output data into a joblib file. ## `warn_about_unused_dependencies(requirements_list)` Raises a warning if some declared dependencies were not used during test execution This method compares the imported modules after test execution with the dependency list in the package's requirements.txt file. If the requirements contains more than what is required for execution, it raises a warning. Source code in `docs/industrial-ai-suite/sdk/simaticai/testing/run_component.py` ```python def warn_about_unused_dependencies(requirements_list): """ Raises a warning if some declared dependencies were not used during test execution This method compares the imported modules after test execution with the dependency list in the package's requirements.txt file. If the requirements contains more than what is required for execution, it raises a warning. """ imported_packages = [] packages = importlib.metadata.packages_distributions() for module_name in sys.modules: try: if module_name and '.' not in module_name: # Only check top-level modules dist = importlib.metadata.distribution(module_name) imported_packages.append(dist.name) except Exception: if module_name in packages: pkgs = packages.get(module_name) for pkg in pkgs: imported_packages.append(pkg) imported_packages = set([pkg.replace('-','_').lower() for pkg in imported_packages]) requirements_list = set([pkg.replace('-','_').lower() for pkg in requirements_list]) diff = requirements_list.difference(imported_packages) if 0 < len(diff): _logger.warning(f"WARNING! The following dependencies were not used during execution: {', '.join(diff)}. Consider removing them from the pipeline package.") ``` ## `main(args)` Feeds input to the entrypoint and captures output. Imports entrypoint module given with its name, and triggers its `run(...)` function with the prepared data in the input file. If pipeline_parameters dictionary is not empty, before triggering `run(..)` method, the `update_parameters(..)` method of entrypoint will be called with the dictionary. The input file must be a joblib dump, and the joblib must be a dictionary or list of dictionaries. One dictionary represents one input for the component with the required variable names and values, which is directly passed to `run()`. The output file is a dumped joblib result which is a list containing outputs of the component, in the structure returned from `run()`. Parameters: | Name | Type | Description | Default | | ------ | ----------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------- | | `args` | `Namespace` | Command line arguments module_name (str): Name of the entrypoint Python script input_file (os.Pathlike): Path of the joblib file containing the input payloads output_file (os.Pathlike): Path of the joblib file where the outputs will be stored pipeline_parameters (json-string): json formatted dictionary defining configurable parameters with their names as key and their values | *required* | Source code in `docs/industrial-ai-suite/sdk/simaticai/testing/run_component.py` ```python def main(args: argparse.Namespace) -> int: """ Feeds input to the entrypoint and captures output. Imports entrypoint module given with its name, and triggers its `run(...)` function with the prepared data in the input file. If pipeline_parameters dictionary is not empty, before triggering `run(..)` method, the `update_parameters(..)` method of entrypoint will be called with the dictionary. The input file must be a joblib dump, and the joblib must be a dictionary or list of dictionaries. One dictionary represents one input for the component with the required variable names and values, which is directly passed to `run()`. The output file is a dumped joblib result which is a list containing outputs of the component, in the structure returned from `run()`. Args: args (argparse.Namespace): Command line arguments
module_name (str): Name of the entrypoint Python script
input_file (os.Pathlike): Path of the joblib file containing the input payloads
output_file (os.Pathlike): Path of the joblib file where the outputs will be stored
pipeline_parameters (json-string): json formatted dictionary defining configurable parameters with their names as key and their values """ entrypoint = importlib.import_module(args.module_name) trigger_method = None try: inspect.signature(entrypoint.process_input) trigger_method = "process_input" except AttributeError: try: inspect.signature(entrypoint.run) trigger_method = "run" except AttributeError: _logger.warning("Method run not found") if trigger_method is None: raise RuntimeError("Neither 'run(data: str)' nor 'process_input(data: dict)' entrypoint method can be found.") if args.pipeline_parameters is not None and args.pipeline_parameters != "{}": pipeline_parameters = json.loads(args.pipeline_parameters) else: pipeline_parameters = {} try: _logger.debug(f"Calling `update_parameters(..)` with: {args.pipeline_parameters}") entrypoint.update_parameters(pipeline_parameters) except AttributeError: _logger.warning("Entrypoint does not implement `update_parameters()` method. Skipping pipeline parameter update.") input_list = joblib.load(args.input_file) if not isinstance(input_list, list): raise ValueError("Component input must be supplied as a list.") if trigger_method == "process_input": _logger.debug("Calling `process_input(..)`") else: _logger.debug("Calling `run(..)`") output_list = [] for input_data in input_list: if trigger_method == "process_input": output_list.append(entrypoint.process_input(input_data)) else: output_list.append(entrypoint.run(json.dumps(input_data))) if args.requirements_file is not None: requirements_list = Path(args.requirements_file).read_text().split('#') warn_about_unused_dependencies(requirements_list) joblib.dump(output_list, args.output_file) if trigger_method == "process_input": return 0 else: _logger.warning("Trigger method `run(data: str)` is deprecated and will be removed in the future. Please refer the user manual.") return 0b10001 # return code 0b10001 means deprecated `run` method was triggered ``` Utility script for running an ONNX model in a given virtual Python environment. It is designed to be executed from `simaticai.testing.PipelineRunner` class. It consumes input data from a joblib file and produces output data into a joblib file. ## `main(model_path, config_path, input_file, output_file)` Feeds input to the ML Model saved in ONNX format and captures output. Reads the given model and creates an onnxruntime Session to The input file must be a joblib dump, and the joblib must be a dictionary or list of dictionaries. The output file is a dumped joblib result list containing the input dictionary extended with the generated predictions. Parameters: | Name | Type | Description | Default | | ------------- | ---------- | -------------------------------------------------------- | ---------- | | `model_path` | `str` | File path for the stored ML Model in ONNX format | *required* | | `input_file` | `Pathlike` | Path of the joblib file containing the input payloads | *required* | | `output_file` | `Pathlike` | Path of the joblib file where the outputs will be stored | *required* | Source code in `docs/industrial-ai-suite/sdk/simaticai/testing/run_gpuruntime_component.py` ```python def main(model_path, config_path, input_file, output_file): """Feeds input to the ML Model saved in ONNX format and captures output. Reads the given model and creates an onnxruntime Session to The input file must be a joblib dump, and the joblib must be a dictionary or list of dictionaries. The output file is a dumped joblib result list containing the input dictionary extended with the generated predictions. Args: model_path (str): File path for the stored ML Model in ONNX format input_file (os.Pathlike): Path of the joblib file containing the input payloads output_file (os.Pathlike): Path of the joblib file where the outputs will be stored """ input_list = joblib.load(input_file) input_list = input_list if type(input_list) is list else [input_list] output_list = [] session = InferenceSession(model_path) model_config = _get_proto_config(config_path) max_batch_size = model_config.get("maxBatchSize", 0) inputs = [input for input in session.get_inputs()] input_names = [input.name for input in inputs] outputs = [output for output in session.get_outputs()] output_names = [output.name for output in outputs] for _input in input_list: _input_tensor = {} output = {k: _input.get(k) for k in _input.keys() if k not in input_names} for _input_info in inputs: if max_batch_size > 0: input_shape = _get_new_shape(model_config["input"], _input[_input_info.name], _input_info.name) if input_shape[0] > max_batch_size: _logger.warning(f"Received input batch size ({input_shape[0]}) is greater than max_batch_size ({max_batch_size})!") _input_tensor[_input_info.name] = _input[_input_info.name].reshape(input_shape) else: _input_tensor[_input_info.name] = _input[_input_info.name].reshape(_input_info.shape) model_outputs = session.run(output_names, _input_tensor) output |= dict(zip(output_names, model_outputs)) output_list.append(output) joblib.dump(output_list, output_file) return 0 ``` RunnerConfig module and class is responsible to handle the configuration json.\ An instance of RunnerConfig class is being used to setup a LocalPipelineRunner with the required parameters. ## `TimeSeriesStreamConfig` TimeSeriesStreamConfig is a configuration class used to manage settings for the time series stream. Attributes: fields (list): List of fields to be included in the time series stream. Defaults to None. count (int): Number of records to be included in the time series stream. Defaults to None. offset (int): Offset to be applied in the time series stream. Defaults to None. batch_size (int): Size of the batch to be used in the time series stream. Defaults to None. Source code in `docs/industrial-ai-suite/sdk/simaticai/testing/runner_config.py` ```python @dataclass class TimeSeriesStreamConfig: """ TimeSeriesStreamConfig is a configuration class used to manage settings for the time series stream. Attributes: fields (list): List of fields to be included in the time series stream. Defaults to None. count (int): Number of records to be included in the time series stream. Defaults to None. offset (int): Offset to be applied in the time series stream. Defaults to None. batch_size (int): Size of the batch to be used in the time series stream. Defaults to None. """ fields: list = field(default=None) count: int = field(default=None) offset: int = field(default=None) batch_size: int = field(default=None) ``` ## `VCAStreamConfig` VCAStreamConfig is a configuration class used to manage settings for the VCA stream. Attributes: variable_name (str): Name of the variable to be used in the VCA stream. Defaults to None. image_format (str): Format of the images to be used in the VCA stream. Defaults to None. filter (str): Filter to be applied in the VCA stream. Defaults to None. Source code in `docs/industrial-ai-suite/sdk/simaticai/testing/runner_config.py` ```python @dataclass class VCAStreamConfig: """ VCAStreamConfig is a configuration class used to manage settings for the VCA stream. Attributes: variable_name (str): Name of the variable to be used in the VCA stream. Defaults to None. image_format (str): Format of the images to be used in the VCA stream. Defaults to None. filter (str): Filter to be applied in the VCA stream. Defaults to None. """ variable_name: str = field(default=None) image_format: str = field(default=None) filter: str = field(default=None) ``` ## `RunnerConfig` RunnerConfig is a configuration class used to manage settings for running tests and streams. Attributes: data (PathLike): Path to the data file or directory. Defaults to None. test_dir (PathLike): Path to the test directory. Defaults to None. cleanup (bool): Indicates whether to clean up resources after execution. Defaults to True. time_series_stream (TimeSeriesStreamConfig): Configuration for the time series stream. vca_stream (VCAStreamConfig): Configuration for the VCA stream. Methods: **post_init**(): Converts `data` and `test_dir` attributes to `Path` objects if they are not None. **dict**(): Returns a dictionary representation of the configuration, including nested stream configurations. from_json(config_path: PathLike | str) -> RunnerConfig: Creates a `RunnerConfig` instance from a JSON configuration file. Args: config_path (PathLike | str): Path to the JSON configuration file. Returns: RunnerConfig: An instance of the `RunnerConfig` class populated with values from the JSON file. Exceptions: - Logs a warning and returns a default `RunnerConfig` instance if the file is not found. - Logs a warning and returns a default `RunnerConfig` instance if the file is not a valid JSON. - Logs a warning and returns a default `RunnerConfig` instance if any other error occurs during file reading. Source code in `docs/industrial-ai-suite/sdk/simaticai/testing/runner_config.py` ```python @dataclass class RunnerConfig: """ RunnerConfig is a configuration class used to manage settings for running tests and streams. Attributes: data (PathLike): Path to the data file or directory. Defaults to None. test_dir (PathLike): Path to the test directory. Defaults to None. cleanup (bool): Indicates whether to clean up resources after execution. Defaults to True. time_series_stream (TimeSeriesStreamConfig): Configuration for the time series stream. vca_stream (VCAStreamConfig): Configuration for the VCA stream. Methods: __post_init__(): Converts `data` and `test_dir` attributes to `Path` objects if they are not None. __dict__(): Returns a dictionary representation of the configuration, including nested stream configurations. from_json(config_path: PathLike | str) -> RunnerConfig: Creates a `RunnerConfig` instance from a JSON configuration file. Args: config_path (PathLike | str): Path to the JSON configuration file. Returns: RunnerConfig: An instance of the `RunnerConfig` class populated with values from the JSON file. Exceptions: - Logs a warning and returns a default `RunnerConfig` instance if the file is not found. - Logs a warning and returns a default `RunnerConfig` instance if the file is not a valid JSON. - Logs a warning and returns a default `RunnerConfig` instance if any other error occurs during file reading. """ data: PathLike = field(default=None) test_dir: PathLike = field(default=None) cleanup: bool = field(default=True) time_series_stream: TimeSeriesStreamConfig = field(default_factory=TimeSeriesStreamConfig) vca_stream: VCAStreamConfig = field(default_factory=VCAStreamConfig) def __post_init__(self): if self.data is not None: self.data = Path(self.data) if self.test_dir is not None: self.test_dir = Path(self.test_dir) @property def __dict__(self): return { "data": self.data, "test_dir": self.test_dir, "cleanup": self.cleanup, "TimeSeriesStream": { "fields": self.time_series_stream.fields, "count": self.time_series_stream.count, "offset": self.time_series_stream.offset, "batch_size": self.time_series_stream.batch_size }, "VCAStream": { "variable_name": self.vca_stream.variable_name, "image_format": self.vca_stream.image_format, "filter": self.vca_stream.filter } } @classmethod def from_json(cls, config_path: PathLike | str): import json try: config_path = Path(config_path) with open(config_path, 'r') as file: json_config = json.load(file) except FileNotFoundError: logger.warning(f"Configuration file {config_path} does not exist.") return cls() except json.JSONDecodeError: logger.warning(f"Configuration file {config_path} is not a valid JSON.") return cls() except Exception as e: logger.warning(f"An error occurred while reading the configuration file {config_path}: {e}") return cls() data = json_config.get('data') test_dir = json_config.get('test_dir') cleanup = json_config.get('cleanup', True) time_series_stream = TimeSeriesStreamConfig( fields=json_config.get('TimeSeriesStream', {}).get('fields'), count=json_config.get('TimeSeriesStream', {}).get('count'), offset=json_config.get('TimeSeriesStream', {}).get('offset'), batch_size=json_config.get('TimeSeriesStream', {}).get('batch_size') ) vca_stream = VCAStreamConfig( variable_name=json_config.get('VCAStream', {}).get('variable_name'), image_format=json_config.get('VCAStream', {}).get('image_format'), filter=json_config.get('VCAStream', {}).get('filter') ) return cls( data=data, test_dir=test_dir, cleanup=cleanup, time_series_stream=time_series_stream, vca_stream=vca_stream, ) ``` ## `TimeSeriesStream` Bases: `DataStream` This class creates a generator from a csv file. The generate function returns a generator that reads the csv file line by line and converts each line to an input dictionary, as if it were received from AI Inference Server. Source code in `docs/industrial-ai-suite/sdk/simaticai/testing/timeseries_stream.py` ```python class TimeSeriesStream(DataStream): """ This class creates a generator from a csv file. The generate function returns a generator that reads the csv file line by line and converts each line to an input dictionary, as if it were received from AI Inference Server. """ def __init__(self, csv_path: os.PathLike, *, fields: Optional[list] = None, count: Optional[int] = None, offset: Optional[int] = None, batch_size: Optional[int] = None): """ Creates a new TimeSeriesStream object Args: csv_path (os.Pathlike): Path to the csv file. fields (Optional[list[str]]): List of required column headers. Will use all columns when None. """ self.csv_path = Path(csv_path) if fields is None: self.fields = [] else: self.fields = fields with open(self.csv_path, "r") as file: self.header = file.readline().strip().split(",") if any(key for key in self.fields if key not in self.header): raise KeyError("The CSV file must contain variable names in the first row.") if self.fields == []: self.fields = self.header fields_len = len(self.fields) filtered_len = len(list(k for k in self.fields if re.match("[a-zA-Z]+", k))) if filtered_len != fields_len: raise KeyError("Column headers should start with a letter.") if (not isinstance(count, int)) or count < 0: self.count = 0 else: self.count = count if (not isinstance(offset, int)) or offset < 0: self.offset = 0 else: self.offset = offset if (not isinstance(batch_size, int)) or batch_size < 0: self.batch_size = 0 else: self.batch_size = batch_size def _read_value(self, value): try: return int(value) except ValueError: try: return float(value) except ValueError: return value def _read_csv(self): with open(self.csv_path, 'r', encoding='UTF-8') as csv_file: csv_reader = csv.DictReader(csv_file) for line in csv_reader: if 0 < len(self.fields): key_holder = self.fields else: key_holder = line.keys() result = {} for k in key_holder: result[k] = self._read_value(line[k]) yield result def _limit(self): _max = float('inf') if 0 < self.count: _max = self.offset + self.count counter = 0 for line in self._read_csv(): if _max <= counter: break counter += 1 if counter <= self.offset: continue yield line def _batch(self): aggregate = [] for line in self._limit(): if len(aggregate) < self.batch_size: aggregate += [line] continue yield aggregate aggregate = [line] if self.batch_size == len(aggregate): yield aggregate else: _logger.warning(f"WARNING! The length of the given dataset is not divisible by {self.batch_size}. There are {len(aggregate)} inputs remaining in the buffer.") def __iter__(self): """ Creates the input data generator. Returns: a generator """ if 0 < self.batch_size: return self._batch() else: return self._limit() ``` ### `__init__(csv_path, *, fields=None, count=None, offset=None, batch_size=None)` Creates a new TimeSeriesStream object Parameters: | Name | Type | Description | Default | | ---------- | --------------------- | ---------------------------------------------------------------- | ---------- | | `csv_path` | `Pathlike` | Path to the csv file. | *required* | | `fields` | `Optional[list[str]]` | List of required column headers. Will use all columns when None. | `None` | Source code in `docs/industrial-ai-suite/sdk/simaticai/testing/timeseries_stream.py` ```python def __init__(self, csv_path: os.PathLike, *, fields: Optional[list] = None, count: Optional[int] = None, offset: Optional[int] = None, batch_size: Optional[int] = None): """ Creates a new TimeSeriesStream object Args: csv_path (os.Pathlike): Path to the csv file. fields (Optional[list[str]]): List of required column headers. Will use all columns when None. """ self.csv_path = Path(csv_path) if fields is None: self.fields = [] else: self.fields = fields with open(self.csv_path, "r") as file: self.header = file.readline().strip().split(",") if any(key for key in self.fields if key not in self.header): raise KeyError("The CSV file must contain variable names in the first row.") if self.fields == []: self.fields = self.header fields_len = len(self.fields) filtered_len = len(list(k for k in self.fields if re.match("[a-zA-Z]+", k))) if filtered_len != fields_len: raise KeyError("Column headers should start with a letter.") if (not isinstance(count, int)) or count < 0: self.count = 0 else: self.count = count if (not isinstance(offset, int)) or offset < 0: self.offset = 0 else: self.offset = offset if (not isinstance(batch_size, int)) or batch_size < 0: self.batch_size = 0 else: self.batch_size = batch_size ``` ### `__iter__()` Creates the input data generator. Returns: a generator Source code in `docs/industrial-ai-suite/sdk/simaticai/testing/timeseries_stream.py` ```python def __iter__(self): """ Creates the input data generator. Returns: a generator """ if 0 < self.batch_size: return self._batch() else: return self._limit() ``` This module contains the VCAStream class, which generates image data streams from a specified folder of images. The images are converted into the ImageSet format, suitable for use on testing purposes. The VCAStream class inherits from DataStream and provides an iterator that yields ImageSet objects containing images in the specified format. Example usage vca_stream = VCAStream(data='/path/to/image/folder', variable_name='vision_payload', image_format='BayerRG8') with LocalPipelineRunner("MyPipeline-edge_1.zip") as runner: runner.run_pipeline(vca_stream) ## `VCAStream` Bases: `DataStream` This class creates a generator from a folder of images. The generate function returns a generator that walks over the image folder and converts each image into the specified format, BayerRG8 by default. The resulting object is in the ImageSet format, as if it were received from AI Inference Server. Source code in `docs/industrial-ai-suite/sdk/simaticai/testing/vca_stream.py` ```python class VCAStream(DataStream): """ This class creates a generator from a folder of images. The generate function returns a generator that walks over the image folder and converts each image into the specified format, BayerRG8 by default. The resulting object is in the ImageSet format, as if it were received from AI Inference Server. """ def __init__(self, data: os.PathLike, variable_name: str = 'vision_payload', image_format: str | ImageFormat = 'BayerRG8', filter: str | None = None): """ Creates a new VCAStream object Args: data (os.Pathlike): Path to the directory of images variable_name (str): Name of the variable to store the images (default: 'vision_payload') image_format (str): Supported image formats: 'BGR' (equivalently: 'BGR8'), 'RGB' (equivalently: 'RGB8'), 'BayerRG8' (default), 'BayerGR8', 'BayerBG8', 'BayerGB8', 'Mono8', 'YUV422Packed', 'YUV422_YUYV_Packed' filter (rglob_pattern): Pattern to filter the images (see also: pathlib.rglob()) """ self.seq = 0 self.data = data if filter is None or "" == filter.strip(): self.filter = "**/*.[jJpP][pPnN][gGeE]*" else: self.filter = filter if variable_name is None or "" == variable_name.strip(): self.variable_name = 'vision_payload' else: self.variable_name = variable_name if image_format is None: self.image_format = ImageFormat.BayerRG8 else: try: self.image_format = ImageDetails._parse_image_format(image_format) except ValueError as err: raise AssertionError(str(err)) self.camera_id = uuid.uuid4() self._iter_image_paths: list[Path] = [] self._iter_current_index: int = 0 def __iter__(self): """ Initializes the iterator by collecting all image paths and resetting the index. Returns: self (the iterator object) """ self._iter_image_paths = list(Path(self.data).rglob(self.filter)) self._iter_current_index = 0 return self def __next__(self): """ Returns the next ImageSet from the image folder. Raises: StopIteration: When there are no more images to process. Returns: A dictionary containing the ImageSet variable. """ if len(self._iter_image_paths) == 0: self.__iter__() if self._iter_current_index >= len(self._iter_image_paths): raise StopIteration image_path = self._iter_image_paths[self._iter_current_index] self._iter_current_index += 1 return self._create_imageset(image_path) def _create_imageset(self, image_path): image_detail = ImageDetails.from_image(image_path=image_path, format=self.image_format) image_detail.id = f'VCA Stream : {image_detail.id}' image_detail.metadata = {"ptpstatus": "Disabled", "ptptimestamp": "0"} image_detail.seq = self.seq self.seq += 1 imageset = ImageSet( version='1', cameraid=str(self.camera_id), timestamp=image_detail.timestamp, ) imageset.add_image(image_detail) result = {self.variable_name: imageset.to_dict()} return result ``` ### `__init__(data, variable_name='vision_payload', image_format='BayerRG8', filter=None)` Creates a new VCAStream object Parameters: | Name | Type | Description | Default | | --------------- | --------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------ | | `data` | `Pathlike` | Path to the directory of images | *required* | | `variable_name` | `str` | Name of the variable to store the images (default: 'vision_payload') | `'vision_payload'` | | `image_format` | `str` | Supported image formats: 'BGR' (equivalently: 'BGR8'), 'RGB' (equivalently: 'RGB8'), 'BayerRG8' (default), 'BayerGR8', 'BayerBG8', 'BayerGB8', 'Mono8', 'YUV422Packed', 'YUV422_YUYV_Packed' | `'BayerRG8'` | | `filter` | `rglob_pattern` | Pattern to filter the images (see also: pathlib.rglob()) | `None` | Source code in `docs/industrial-ai-suite/sdk/simaticai/testing/vca_stream.py` ```python def __init__(self, data: os.PathLike, variable_name: str = 'vision_payload', image_format: str | ImageFormat = 'BayerRG8', filter: str | None = None): """ Creates a new VCAStream object Args: data (os.Pathlike): Path to the directory of images variable_name (str): Name of the variable to store the images (default: 'vision_payload') image_format (str): Supported image formats: 'BGR' (equivalently: 'BGR8'), 'RGB' (equivalently: 'RGB8'), 'BayerRG8' (default), 'BayerGR8', 'BayerBG8', 'BayerGB8', 'Mono8', 'YUV422Packed', 'YUV422_YUYV_Packed' filter (rglob_pattern): Pattern to filter the images (see also: pathlib.rglob()) """ self.seq = 0 self.data = data if filter is None or "" == filter.strip(): self.filter = "**/*.[jJpP][pPnN][gGeE]*" else: self.filter = filter if variable_name is None or "" == variable_name.strip(): self.variable_name = 'vision_payload' else: self.variable_name = variable_name if image_format is None: self.image_format = ImageFormat.BayerRG8 else: try: self.image_format = ImageDetails._parse_image_format(image_format) except ValueError as err: raise AssertionError(str(err)) self.camera_id = uuid.uuid4() self._iter_image_paths: list[Path] = [] self._iter_current_index: int = 0 ``` ### `__iter__()` Initializes the iterator by collecting all image paths and resetting the index. Returns: self (the iterator object) Source code in `docs/industrial-ai-suite/sdk/simaticai/testing/vca_stream.py` ```python def __iter__(self): """ Initializes the iterator by collecting all image paths and resetting the index. Returns: self (the iterator object) """ self._iter_image_paths = list(Path(self.data).rglob(self.filter)) self._iter_current_index = 0 return self ``` ### `__next__()` Returns the next ImageSet from the image folder. Raises: | Type | Description | | --------------- | ----------------------------------------- | | `StopIteration` | When there are no more images to process. | Source code in `docs/industrial-ai-suite/sdk/simaticai/testing/vca_stream.py` ```python def __next__(self): """ Returns the next ImageSet from the image folder. Raises: StopIteration: When there are no more images to process. Returns: A dictionary containing the ImageSet variable. """ if len(self._iter_image_paths) == 0: self.__iter__() if self._iter_current_index >= len(self._iter_image_paths): raise StopIteration image_path = self._iter_image_paths[self._iter_current_index] self._iter_current_index += 1 return self._create_imageset(image_path) ``` # Product Excellence Program Siemens collects information about the use of our products by its users. This information will help us to improve our product features and functionality to better meet our customers’ needs. All data collected under the Product Excellence Program will be aggregated and pseudonymized. No intellectual property information is collected or sent. Product performance is not affected. The details regarding the types of data and the third parties engaged are described in the [Siemens Trust Center](https://www.siemens.com/global/en/company/about/compliance/dataprivacy/product-excellence-program.html). If you wish to control the participation (or withdraw any given consent) with regard to the data collection under the Product Excellence Program, you can do so by calling `simaticai telemetry`. ## What happens when you give your consent? During the building of the Pipeline Package: - AI SDK - collects the information below - creates a telemetry_data.yml file and attaches to the Pipeline Package - AI Inference Server - reads the information during Pipeline Import - sends the data to the Server anonymously Collected data may include: - environment info (CI/CD) - python version - simaticai version - pipeline file extension - platform os - files types included in the created Pipeline Package - usage of LocalPipelineRunner Personal data (e.g., names, email addresses, passwords) will not be collected. ## Disclaimer By deploying the created pipeline packages to AI Inference Server, the user accepts the terms of the previously provided consent and agrees to the transmission of the previously collected telemetry data about the packaging and testing environment. Siemens is not responsible for either the authenticity of the content of the consent or the collected data. # Vision Connector App Vision Connector allows connectivity and image acquisition from single or multiple industrial cameras on Siemens Industrial Edge. Do you want to gain a competitive advantage for your vision use cases? Does your use case require connecting to one or multiple cameras on the shop floor? Vision Connector gives you the advantage of connecting to industrial Ethernet cameras and transferring the acquired data via high-speed interface to Siemens Industrial Edge applications. The acquired image data can be ingested by AI Inference Server or other Siemens Industrial Edge applications. The integrated UI and convenience of setting up the camera connectivity can open up many opportunities for your vision use cases. ## Key features and benefits - Key Features - Connect to one or more Industrial Ethernet cameras on Siemens Industrial Edge - Offers high-speed ZMQ interface for data ingestion to Industrial Edge applications - Easily add, remove, enable and disable cameras on the UI - Preview image streams on the UI - Camera auto discovery to locate new cameras in the network - Conveniently adjust camera acquisition parameters such as: Frequency, Resolution, Pixel Format, Exposure time, Height, Width - Support PTP acquisition from Cameras - Export/Import configuration file with cameras and parameters - Integrated log viewer on UI for troubleshooting - Benefits - Customers can connect industrial cameras to Industrial Edge - Customers do not require to build their own custom connector - Provides standard interface and payload definition for IE apps - Provides integration with AI Inference Server ## Available APIs | OpenAPIs | | ------------------------------------------------ | | [Vision Connector App OpenAPI](openapi/api.html) | # Vision Connector App OpenAPI [Download OpenAPI Specification](openapi.json) View the OpenAPI specification # Vision Data Collector ## Are you searching for a standard and convenient way to collect vision data for your AI solution? Vision Data Collector is a Siemens Industrial Edge App which captures images from Vision Connector, inference results from AI Inference Server, and stores them on a file server. You can use the captured data to (re)train and diagnose AI models. ## Key features and benefits - Key features - Collect images from Vision Connector via ZeroMQ - Collect images from the shopfloor according to defined rules - Store those images locally (file system) - Collect prediction results for those images and store them locally - Support image conversion and image resizing - Provide data for visualization on Edge via WebSocket API - Send collected data to the external environment (pull/push mechanism) according to the defined rules (FTP/STFP, Customer Endpoint) - Allows custom sync to Cloud (e.g. MDF, FlowCreator, Kafka) - Simple GUI for system setup, rules configuration and logs - Benefits - Standard and convenient way to collect vision data for (re)training. - Close the MLOps loop for IAI Portfolio. - Enables additional rich functions with image metadata, such as inference result visualization/calibration, automatic image labeling, data filtering, etc.. - Collect images and inference result in a configurable way. - Store data in local file system with customized storage volume. - Provide data flexibly (local export, enterprise-level server storage, to cloud, etc.). ## Available APIs | OpenAPIs | | ------------------------------------------------- | | [Vision Data Collector OpenAPI](openapi/api.html) | # Vision Data Collector OpenAPI [Download OpenAPI Specification](VDC_api_spec.json) View the OpenAPI specification # Industrial Asset Hub documentation # Contact ## Devices and Marketplace - [Industrial Assets Marketplace](https://www.dex.siemens.com/?selected=assets) ## Contact Us Directly Let our experts advise you! If you want to know more about Industrial Edge and potential applications in your company, just [talk to our experts](mailto:xo-cdm-devops.industry@siemens.com) – they’ll be happy to help. Request a demo, report an issue, request access through our [service desk](https://agileworld.siemens.cloud/jira/servicedesk/customer/portal/42) - [Overview](overview.html) - [Industrial Asset Hub APIs](/apis.html?productLine=Industrial+Asset+Hub) - [Contact](contact.html) # Industrial Asset Hub Siemens Industrial Asset Hub (IAH) simplifies industrial asset management by combining proven device management methods in a scalable, open, and secure cloud service that discovers, identifies, and manages industrial assets across industrial enterprises, regardless of vendor or type. Figure: OT management big picture ## How does Industrial Asset Hub work IAH enables the users to discover all OT devices that can be reached from a gateway device in the industrial producer’s respective on-site installation. The discovery happens independent of device type and vendor where the IAH service reports status and inventory information in the device specific format through a common and stable API as well a cloud-based web UI centrally to the user. After the discovery, the users can identify devices via respective Asset Link based on comprehensive device and status information. These device and status information comprise vendor, device type (e.g., PLC, HMI, network router) order number, serial number, MAC addresses of the network interfaces as well as software, firmware, and hardware versions. All information is collected in the centralized inventory list that can be manually extended by the user with custom fields. The inventory list is accessible using IAH’s web dashboard application but can as well be integrated seamlessly into any dashboard solutions, central IT asset management systems, or other applicable systems by utilizing its integrated and well-documented API. This way, IAH users gain transparency of device inventory and have access to the device state information and installed firmware versions at anytime and anywhere in the world, just by accessing the right APIs. ## Industrial Asset Hub Building Blocks Industrial Asset Hub (IAH) consists of three focal components (technical view): 1. A centrally hosted cloud-service (SaaS, also referred to as “backend service”), encompassing and exposing all capabilities to support the users. These capabilities can be reached via a web-based UI application and well-documented and openly accessible APIs. 1. A gateway software (also referred to as Asset Gateway), which is hosted on premise to route information and servicing functions to the backend service. The communication between the Asset Gateway and backend service is established with firewall-friendly protocols. 1. A selection of Asset Links (also referred to as “ALs”), which act as middleware, between the Asset Gateway and the field devices. The Asset Gateway and one or more Asset Links are hosted on Industrial Edge devices, SCALANCE LPE devices, or as docker-compose on any machine which can run containers. ## Developer Resources Learn how to use the Industrial Asset Hub and start integrating it with your enterprise services. Create solutions on top of the Industrial Asset Hub database. | Link | Description | | ----------------------------------------------------------------------------------------------------------------- | ------------------------ | | [Developer Documentation](https://industrial-assets.io/developers/programing-interface/programing-interface.html) | Developer Documentation. | # Industrial Edge documentation # Contact ## Playground Registration [Industrial Edge Playground registration](https://www.siemens.com/global/en/products/automation/topic-areas/industrial-edge/developer/playground-registration.html) - Are you ready to try out how Industrial Edge Management system, edge devices and edge apps interact for best-in-class data visualization? Just fill out the form to receive your access data! ## Community Your forum for information, knowledge exchange and solutions around Industrial Edge. Ask questions, post ideas, and stay up-to-date on Industrial Edge news. [Join the conversation](https://forum.mendix.com/link/space/industrial-edge) ## Devices and Marketplace - [Industrial Edge Devices](https://www.siemens.com/global/en/products/automation/topic-areas/industrial-edge/industrial-edge-devices.html) - [Industrial Edge Marketplace](https://www.dex.siemens.com/?selected=edge) ## Training/Learning Become an Industrial Edge expert with our [free, virtual training courses](https://siemens-learning-simaticedge.sabacloud.com/Saba/Web_spf/EU2PRD0112/app/workspace/detail/pgcnt000000000518589). Check out the Edge App Development Curriculum here. If it is not displayed directly, please search for “0000057309”. ## Contact Let our experts advise you! If you want to know more about Industrial Edge and potential applications in your company, just [talk to our experts](https://www.siemens.com/global/en/products/automation/topic-areas/industrial-edge.html#Letourexpertsadviseyou) – they’ll be happy to help. # Industrial Edge Open Source Projects Industrial Edge is the open Industrial Edge Computing platform from Siemens and part of Siemens’ comprehensive Industrial IoT offering Below is the list of all [Industrial Edge Open Source Projects](https://github.com/industrial-edge/). - [Overview](overview.html) - Industrial Edge APIs - [Databus Common Payload format](api/databus/overview.html) - [Changelog](api/databus/changelog.html) - [Async API Specification](api/databus/databus-async-api.html) - [v1.4.0](api/databus/v1.4.0/asyncapi.html) - [v1.3.0](api/databus/v1.3.0/asyncapi.html) - [v1.2.2](api/databus/v1.2.2/asyncapi.html) - [PROFINET IO Connector v1.2.0](api/databus/profinet_v1.2.0/asyncapi.html) - [Flow Creator Configurator](api/flowcreator_configurator/overview.html) - Flow Creator Configurator API - [v2.0.0](api/flowcreator_configurator/v2.0.0/flowcreator-configurator-api-2.0.0.html) - [v1.1.0](api/flowcreator_configurator/v1.1.0/flowcreator-configurator-api-1.1.0.html) - [V1.0.0](api/flowcreator_configurator/v1.0.0/flowcreator-configurator-api-1.0.0.html) - [Profinet Asset Service](api/Profinet%20Asset%20Service/overview.html) - Profinet Asset Service API - [v1.0.1 Client](api/Profinet%20Asset%20Service/v1.0.1/Client%20Proto/Profinet%20Client%20Proto%20overview.html) - [v1.0.1 Server](api/Profinet%20Asset%20Service/v1.0.1/Server%20Proto/Profinet%20Server%20Proto%20overview.html) - [External Databus](api/external_databus/overview.html) - External Databus API - [V1.0.0](api/external_databus/v1.0.0/external_databus-api-1.0.0.html) - [IE Device](api/ied/overview.html) - IE Device API - [V2.5.0](api/ied/ied-api/v2.5.0/ied-api-2.5.0.html) - [V2.4.2](api/ied/ied-api/v2.4.2/ied-api-2.4.2.html) - [V2.4.1](api/ied/ied-api/v2.4.1/ied-api-2.4.1.html) - [V2.4.0](api/ied/ied-api/v2.4.0/ied-api-2.4.0.html) - [V2.1.0](api/ied/ied-api/v2.1.0/ied-api-2.1.0.html) - [V2.0.0](api/ied/ied-api/v2.0.0/ied-api-2.0.0.html) - [V1.0.0](api/ied/ied-api/v1.0.0/ied-api-1.0.0.html) - IE Device App Roles API - [V1.0.0](api/ied/ied-app-roles-api/v1.0.0/ied-app-roles-api-1.0.0.html) - IE Device OIDC API - [V1.0.0](api/ied/ied-oidc-api/v1.0.0/ied-oidc-api-1.0.0.html) - IE Device User API - [V1.0.0](api/ied/ied-user/v1.0.0/ied-user-api-1.0.0.html) - Secure Storage - [V2.0.0](api/ied/secure-storage/v2.0.0/secure-storage-api-2.0.0.html) - Service Registry - [V2.0.0](api/ied/service_registry/RestAPIs.html) - [IE Management](api/iem/overview.html) - Admin API - [V1.0.0](api/iem/api-admin/v1.0.0/api-admin-1-0-0.html) - Portal API - [V1.9.0](api/iem/api-portal/v1.9.0/api-portal-1-9-0.html) - [IE Management V2](api/iemV2/overview.html) - Portal API - [V2.0.0](api/iemV2/api-portal/api-portal-2-0-0.html) - [Vision Payload Specification](api/vision-payload-specification/overview.html) - [V1.1.2](api/vision-payload-specification/v1.1.2/vision-payload-specification.html) - [V1.1.0](api/vision-payload-specification/v1.1.0/vision-payload-specification.html) - [V1.0.0](api/vision-payload-specification/v1.0.0/vision-payload-specification.html) - [IIH APIs](api/iih/overview.html) - Guides - [Secure Storage](api/how-to/secure-storage.html) - [Industrial Edge Mendix Resources](/apis.html?resourceType=Low+Code&query=Industrial+Edge) - [Open Source Projects](github.html) - [Contact](contact.html) # Industrial Edge ## What is Industrial Edge? [Siemens Industrial Edge](https://www.siemens.com/global/en/products/automation/topic-areas/industrial-edge.html) (IE) is an open, ready-to-use edge computing platform, with edge devices, edge applications (apps) and connectivity through an integrated app and device management infrastructure. It allows you to easily connect, manage and operate globally distributed edge installations, with your own or the diverse apps available through the [Industrial Edge Marketplace](https://www.dex.siemens.com/?selected=edge). ## Industrial Edge Ecosystem The Industrial Edge Ecosystem is for your implementation and usage, if you are an industrial producer or machine producer. You can connect, create, deploy and scale your industrial use case using the Industrial Edge platform in combination with apps, devices and solutions from the Industrial Edge Ecosystem. You may also develop your own apps and deploy them via the Industrial Edge Management. ## Industrial Edge Devices A wide range of Industrial Edge devices makes it easy to deploy edge computing in your production environment: You can use SIMATIC controllers and HMIs with integrated edge functionality, use IPCs and virtualized systems, and obtain certified edge devices from our Industrial Edge Ecosystem partners. [Industrial Edge Device website](https://www.siemens.com/global/en/products/automation/topic-areas/industrial-edge/industrial-edge-devices.html) ## Industrial Edge Apps Customers can develop and deploy their own apps, or buy ready-made apps and existing connectivity from the catalogue on the Industrial Edge Marketplace. These apps from Siemens and Ecosystem partners work for more than 90 % of automation systems. Find all available Industrial Edge apps here: [Industrial Edge Marketplace](https://www.dex.siemens.com/?selected=edge) ## Developer Resources Learn how to use Industrial Edge and start developing your own apps. | Link | Description | | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------- | | [Developer Documentation](https://www.siemens.com/global/en/products/automation/topic-areas/industrial-edge/developer.html) | Developer Documentation. | | [Developer Guide](https://docs.eu1.edge.siemens.cloud/develop_an_application/developer_guide/00_Overview.html) | Introduction to industrial Edge Development | | [Industrial Edge API Reference](https://docs.eu1.edge.siemens.cloud/apis_and_references/index.html) | Industrial Edge APIs and References | | [Industrial Edge Playground Registration](https://www.siemens.com/global/en/products/automation/topic-areas/industrial-edge/developer/playground-registration.html) | Request Access for the Industrial Edge Playground | | [GitHub](https://github.com/industrial-edge) | Industrial Edge on GitHub | | [About Docker](https://docs.docker.com/get-started/docker-overview/) | Docker is the technology to run, deploy and create Industrial Edge apps using containers. | | [Industrial Edge Release Notes]() | Industrial Edge Release Notes | | [SITRAIN Digital Industry Academy](https://www.sitrain-learning.siemens.com/DE/en/rw73738/) | Course: App Development Workflow for Industrial Edge (24 Hours) | ## Device Builder Resources Device Builders can build own device offerings based on Industrial Edge. This includes the development of the device, the integration of the Industrial Edge packages, system tests, support and maintenance of the device offered within the Ecosystem. As a Device Builder, they are partner of the Industrial Edge Ecosystem and can provide added value by selling their Industrial Edge Devices to all participants. | Link | Description | | -------------------------------------------------------------------------------------------------------- | --------------------------- | | [Device Builder Quick Start](https://docs.eu1.edge.siemens.cloud/build_a_device/introduction/index.html) | Device Builder Quick Start. | | [Device Builder Guide](https://docs.eu1.edge.siemens.cloud/build_a_device/device_building/index.html) | Device Builder Guide | ## Device Builder- IE Device Kit API The IE Device Kit API provides the abstraction layer that decouples the Industrial Edge Runtime from the underlying Linux systems. This allows to adapt the runtime and its behavior to serve for the specific needs of different Industrial Edge products. The IE Device Kit API is based on gRPC which provides a modern intermediate process communication style for building distributed applications and microservices. The Industrial Edge platform provides and maintains the protobuf specification files for the APIs contained in the IE Device Kit. These protobuf specifications can be used to create stub implementations for both client and server in various programming languages. The Industrial Edge Runtime ships with a client side implementation of these APIs and expects the host system to provide a server side implementation. | Link | Description | | ---------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | [IE Device Kit System Service](https://github.com/industrial-edge/device-kit-system-service) | System Service is a gRPC & Go based system resource tracker and system controller. | | [IE Device Kit Onboard Service](https://github.com/industrial-edge/device-kit-onboard-service) | Onboard Service is a gRPC & Go based Edge Core activator. This service communicates with all other services and provides the necessary configurations for the Edge Device to successfully onboard to Industrial Edge Management system. | | [IE Device Kit Ntp Service](https://github.com/industrial-edge/device-kit-ntp-service) | NTP Service is a gRPC & Go based NTP configurator microservice for Edge Devices. | | [IE Device Kit Network Service](https://github.com/industrial-edge/device-kit-network-service) | Network Service is a gRPC & Go based network configuration microservice. The network settings of the Edge Devices are configured through this service. | ## Mendix on Edge Mendix on Edge integrates the software development capabilities of Mendix to the Industrial Edge allowing IT and OT people to develop applications with low-code for the shopfloor. With Mendix on Edge we enhance the Industrial Edge offering for every customer developing own applications. Providing an integrated low-code development tool with Mendix on Edge aims to support development of own apps (Make vs. Buy scenario) even for users without extensive IT knowledge. Low-code on the shopfloor, speeds up app development, enables local deployment of Mendix apps for decentralized and real time data processing, secure OT operation, secure connectivity and transmission across OT and IT systems. Which components are needed to build Mendix apps for Industrial Edge?¶ In order to push and use Mendix applications in Industrial Edge you need two components: Download and install the [Industrial Edge Plugin for Mendix Studio Pro](https://marketplace.mendix.com/link/component/222639) for free from the Mendix Marketplace. Once installed, follow the wizard to publish your applications to your IE Hub or IE Management system. For each IE Device that runs Mendix applications you need to obtain a [Mendix Device License](https://www.dex.siemens.com/edge/get-started/mendix-device-license?cclcl=en_US) from the Marketplace. The license entitles you to develop, install and run an unlimited number of Mendix applications on one single Industrial Edge Device with a maximum of 10 named users per app instance and a maximum main memory of 4, 8, 16, 32 or 64 GB (RAM), depending on the selected subscription. Learn more about [Mendix on Edge](https://docs.eu1.edge.siemens.cloud/develop_an_application/mendix/overview.html) | Link | Description | | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | --------------------------------------------------------------------------- | | [Mendix on Edge](https://www.siemens.com/global/en/products/automation/topic-areas/industrial-edge/developer/mendix-on-edge.html) | Develop Industrial Edge Apps using the Mendix low-code development platform | | [Mendix Academy](https://academy.mendix.com/link/paths/152/Build-Your-Industrial-Edge-Apps) | Course: Build your Industrial Edge App with Mendix (4 hours) | | [Mendix on Edge Documentation]() | Mendix on Edge Documentation | # Profinet Asset Service | API | Version | | ----------------------------------- | ------------------------------------------------------------------------- | | Profinet Asset Service Client Proto | [V1.0.1](v1.0.1/Client%20Proto/Profinet%20Client%20Proto%20overview.html) | | Profinet Asset Server Service Proto | [V1.0.1](v1.0.1/Server%20Proto/Profinet%20Server%20Proto%20overview.html) | # Profinet Asset Service APIs ## Download Client Proto Files You can download the Proto file for the API using the link below. [Download Client Proto File](conn_suite_registry.proto) # Profinet Asset Service APIs ## Download Server Proto Files You can download the Proto files for the APIs using the links below. - [Download Server common_address Proto Files](common_address.proto) - [Download Server common_datatype Proto Files](common_datatype.proto) - [Download Server common_filters Proto Files](common_filters.proto) - [Download Server common_identifiers Proto Files](common_identifiers.proto) - [Download Server common_operators Proto Files](common_operators.proto) - [Download Server common_variant Proto Files](common_variant.proto) - [Download Server conn_suite_code Proto Files](conn_suite_code.proto) - [Download Server conn_suite_data_generic Proto Files](conn_suite_data_generic.proto) - [Download Server conn_suite_drv_event Proto Files](conn_suite_drv_event.proto) - [Download Server conn_suite_drv_info Proto Files](conn_suite_drv_info.proto) - [Download Server iah_discover Proto Files](iah_discover.proto) # Release Notes This API version is independent of the Industrial Edge releases. ## V1.4.0 - writing of Datapoints - added Datapoint values write response json topic 'ie/d/j/{msgStructureScheme}/{msgStructureSchemeMajorVersion}/{providerAppInstanceId}/dp/wr/resp{dpConnectionNamePath}{dpCollectionNamePath}' and `subDpValueResponseSimaticV1Msg` - `pubDpValueSimaticV1Payload`: added properties `requestId` and `mdHashVer` to allow response and error handling for write requests. - single read of Datapoints (they don't need to be part of any cyclic topic) - added Datapoint values single read request json 'ie/d/j/{msgStructureScheme}/{msgStructureSchemeMajorVersion}/{providerAppInstanceId}/dp/sr/req/{dpConnectionNamePath}{dpCollectionNamePath}' and `pubSingleReadRequestDpValueSimaticV1Msg` - added Datapoint values single read response json 'ie/d/j/{msgStructureScheme}/{msgStructureSchemeMajorVersion}/{providerAppInstanceId}/dp/sr/resp/{dpConnectionNamePath}{dpCollectionNamePath}' and `subSingleReadResponseDpValueSimaticV1Msg` - reading and writing datapoints which are not cyclically published (e.g. recipe data or setpoints) - `nonCyclicDataPoints` has been added to the metadata in `dpMetaConnectionV1` which contains the list of datapoints which are configured in a connector but are not part of any cyclic data topic - You can use the datapoint ids and topic names defined there with single-read and write operations. - updated Databus url to include the name of the docker network : `mqtt://ie-databus.proxy-redirect:1883`. Is is best practice to always use the fully qualified DNS name, otherwise you might run into issues with no_proxy settings which are based on DNS network names. It might also happen that the DNS resolver might contact an external DNS server when the unqualified name is not found within the docker environment, e.g. because an App is currently not running. - fixed naming of `messages/updateRequestMsg` to `messages/pubUpdateRequestMsg`. This should't affect the interface itself, but is is necessary to fix the schema validation script. - allow `dpMetadataSimaticV1` and `diagConnectorStatusMsg` to be `null` for shutdown scenarios ## V1.3.0 - `dpDataTypeV1` : added dataType 'Raw' - update-request: request a connector to republish the latest values. Needed for values which seldomly change. See `ie/c/j/.../updaterequest` and especially `isResponseToUpdateRequest` in `subDpValueSimaticV11TimeSeriesPayload` and in `subDpValueSimaticV1Payload`. - `providerAppInstanceId`: added pattern to allow only a single optional colon to separate between device name and instanceId - `dpConnectionNamePath` and `dpCollectionNamePath`: fixed pattern to not allow '#' and '+' because those characters are not allowed in MQTT topic names and need to be escaped using URL encoding. This also means that '%' needs to be escaped to allow for URL encoding. - `sampleRateType` and `sampleRateHz`: new concept for high frequency data like audio, vibration or motion control to be transmitted with very small overhead. ## V1.2.2 - Fixed typo: the 'statustopic"' property names (with double-quotes at the end) have been changed to 'statustopic' ## V1.2.1 - Fixed the 'Time_Of_Day' and 'LTime_Of_Day' definition. They were mapped to 'LTime', but 'LTime' is a range data type and not a time of day data type. Also, the allowed maximum value is different. The scaling of the values do not change, it is still 100ns resolution which means the values do not change, only the allowed range is reduced to 24 hours. Future connectors shall avoid using 'Time' but using 'LTime' instead. The same applies for 'Time_Of_Day', use 'LTime_Of_Day' instead. - Refined the description of 'providerAppInstanceId'. When aggregating topics from multiple devices, the topic names need to be unique. That is why during that forwarding, a prefix will be added to the Instance Name part of the topic, separated by a colon. For example the Instance Name will be changed from "s7c1" to "device15:s7c1". ## V1.2.0 - Added new 'Status' message type for connector and connection diagnostics - Added extended quality information 'qx' to values - Added 'applicationName' and 'statustopic' to metadata - Added MQTT bindings ('qos' and 'retain') - Added additional 'asyncapi_profinet_io_conn.yaml' API file for the PROFINET IO Connector specific messages to read and write PROFINET Records ## V1.1.1 - Fixed the definition of 'Time', 'Time_Of_Day' and 'LTime_Of_Day' to match the definition of 'LTime'. Actually, only 'LTime' would have been needed and the other definitions are obsolete at the Databus level. On the PLC level, there are different data types but all are normalized to use the same type on the Databus. Future connectors shall avoid using 'Time', 'Time_Of_Day' and 'LTime_Of_Day' and should use 'LTime'. ## V1.1.0 - Use new `ie-databus` Databus hostname (instead of `ie_databus` which can still be used but contains '\_' which is not allowed in DNS names and might cause some problems with libraries) - Added 'timeseries' and 'binarytimeseries' payload formats (used by PROFINET IO Connector) - Added 'hashVersion' to metadata message - Added 'mdHashVer' to the value messages and 'metadataHashVersion' to the metadata message - Added 'accessMode', 'dataPointAddr', 'acquisitionCycleInMs', 'acquisitionMode' to datapoint metadata - Added 'valueRank' and 'arrayDimensions' for array support of values This version contains a bug and has been replaced by V1.1.1. ## V1.0.0 First Version. # Databus Payload Format AsyncApi ## Available Common Payload Format API versions | API version | | ------------------------------ | | [v1.4.0](v1.4.0/asyncapi.html) | | [v1.3.0](v1.3.0/asyncapi.html) | | [v1.2.2](v1.2.2/asyncapi.html) | ## Available PROFINET IO Connector Payload Format versions | API version | | -------------------------------------------------------------------- | | [PROFINET IO v1.2.0](profinet_v1.2.0/asyncapi_profinet_io_conn.yaml) | # Industrial Edge Common Databus Payload Format This document describes the more general concepts about the payload format which didn't fit in the asyncapi.yaml itself. Please check the [change log](changelog.html) for the change history. ## API Definition as a Contract The IE Common Databus Payload Async API spec is a contract which allows many variations. **As long as a connector is fulfilling the contract, it is considered to be compliant, even when it changes its behavior from one version to another one.** For example, numeric values are no longer quoted in the SIMATIC S7 Connector V1.2. The API spec allows both, the quoted and the unquoted variant, but the quoted one is deprecated. This is considered a non-breaking change because both variations are allowed and all clients should be prepared to handle both variants. This means clients need to: - **Implement** against the API (and not against example messages) - **Test** against example messages Some hints about the implementation - Don’t make any assumption about the content of the “id” field (don’t assume you get the tag name there or that the value is numeric) - Don’t make any assumption about the `applicationName`. Products names often change. To identify a connector instance, use the `providerAppInstanceId` which is part of every topic name, e.g. `mbtcp1` in the topic `ie/m/j/simatic/v1/mbtcp1/dp` for the Modbus TCP Connector. On Industrial Edge there is currently no concept for multiple instances of a connector type. When such a concept will be implemented, further instances will be named `mbtcp2`, `mbtcp3`, etc. - Be prepared that numbers might be quoted as strings or might be actual numbers (the quoted variant is deprecated, and all connectors shall switch to the non-quoted numbers) - 64-bit integer values are special because they cannot be exactly represented by the JavaScript 'Number' datatype and many libraries for other languages have the same problem (that's why we use strings to represent such numbers) - There are at least two kinds of payloads for the cyclic values which need to be supported both: bulk (`subDpValueSimaticV1Payload`) and time series (`subDpValueSimaticV11TimeSeriesPayload`). The Databus Gateway for Connectivity Suite connectors uses the time series format, because it is more compact and efficient, the Simatic S7 Connector and other legacy connectors will use the bulk format. - Expect that the OPC UA Connector uses real arrays instead of using pipe-separated values in strings - Timestamps might miss the fractional seconds part, e.g. "2025-11-05T07:16:06" instead of "2025-11-05T07:16:06.182440Z" or can have less than 7 fractional digits like "2025-11-05T07:16:06.182Z". They always need the "Z" at the end to make sure UTC timezone is meant and the "T" to be conformant to ISO8601. - Check the chapters about scope and lifetime of datapoint IDs and names below ## Implementing Subsets Most connectors are based on the Connectivity Suite framework and only support the newer "timeseries" publish type. Old connectors like the SIMATIC S7 Connector user the “bulk” publish type which is considered outdated because it has more overhead. Some connectors, for example the PROFINET IO Connector, only support `timeseries` or `binarytimeseries`. Not all connectors support all data types. That means, only a subset of the possible features is implemented by a specific connector. Each connector needs to document which subsets are implemented and which API version is implemented. This is important for clients if they did not implement all the features, for example the `timeseries` publish type. ## Compatibility Rules When the minor version of the payload schema changes, only optional fields (example: qx) are added to an object and only new alternative objects are added. Many fields are not formally required, because they have been added after V1.1.0, but newer versions of the connectors shall support them (e.g. pubTopic, hashVersion, qx). Some parts of the API spec are marked as deprecated (e.g. encoding numbers as strings). New connectors shall not use any deprecated fields and existing ones shall consider to remove them in newer releases. The “v1” as major version in the topic name means we could add a new version “v2” in parallel but the effort of doing this is too high, so we do everything to avoid that. The performance of notifying all values twice would also be very bad. ## Meaning of the API Version Semantic versioning is being used, that means: - In case of a breaking change, the major version will be changed - Changes in the minor version are optional additions (new fields in existing messages or new message types) The version of the Common Payload API is in principle independent of the IE Platform and IE App versions. That means, when there are no changes for a new Industrial Edge release, there won’t be a new API version. This also means that a connector might still implement according to an older version of the API. ## Metadata Message Handling Metadata messages need to set the `retain=true` flag for the MQTT message. This is needed to make the metadata available to clients which connect at a later time. To know which messages are supported, you need to check which topics are defined in the metadata. For example to know if the write response which was introduced in V1.4.0 is supported, you check for the value of `pubResponseTopic` because you need it anyhow to receive the response. When to publish this message? 1. When the connector starts 1. When the MetaData has changed 1. When the connector has lost the connection to the DataBus and reconnects. This is needed because the DataBus might have been restarted and lost the value. Please refer also to the fields `mdHashVer` and `hashVersion` of the AsyncAPI definition for more information. ## Value Message Handling Value messages should set the `retain=false` flag for the MQTT message. This improves the performance and avoids that clients work with outdated values. There might be some specific connectors for which the `retain=true` flag is reasonable but they should be aware that the retain values are not preserved when the Databus restarts and needs to be refreshed from time to time. ## QoS In case of the local MQTT message bus, the QoS setting shall be always "0" because TCP guarantees that no messages are lost between the publisher and the broker. The additional guarantees of higher QoS values are not existent when no persistent buffering is implemented. `QoS=1` and `QoS=2` add a lot of additional overhead and reduce the throughput. In high load scenarios, instead of guaranteeing message delivery, this instead leads to an earlier collapse of the system because of the higher load and is therefore counterproductive. In case a proxy is between the publisher and the Databus or when they are running on a different machine and the network or machine might fail, the situation might become different. ## Connector Specific Messages Some connectors (e.g. the PROFINET IO Connector) implement additional specific messages which are not part of the common API spec. Those are defined in an extra asyncapi.yaml (e.g. `asyncapi_profinet_io_conn.yaml`). ## Default Values Values which are identical to the default values can be omitted, the client just uses the default value. ## Unexpected Behaviour ### Publish Mode `bulk` In some cases (e.g. when the CPU load is high), the 'bulk' format might contain the same datapoint multiple times from a different point in time. ## Case Sensitive / Case Insensitive Keys of JSON messages are always case sensitive. When it is not explicitly described, all other values are case sensitive. This applies especially for datapoints and data type names as well as for enum values. ## Messages on the Databus The supported messages on the Databus are defined in the 'asyncapi.yaml' file under `channels` Here a condensed overview about the most important messages: ```yaml 'ie/m/j/simatic/v1/{providerAppInstanceId}/dp': description: 'Datapoint Metadata of SIMATIC JSON payload format' 'ie/d/j/simatic/v1/{providerAppInstanceId}/dp/r{dpConnectionNamePath}{dpCollectionNamePath}': description: 'Datapoint values of SIMATIC JSON payload format for cyclic read (subscriptions)' 'ie/d/j/simatic/v1/{providerAppInstanceId}/dp/w{dpConnectionNamePath}{dpCollectionNamePath}': description: 'Datapoint values of SIMATIC JSON payload format for write requests' 'ie/d/j/simatic/v1/{providerAppInstanceId}/dp/wr/resp{dpConnectionNamePath}{dpCollectionNamePath}': description: | Datapoint values of SIMATIC JSON payload format for write responses. Added in version 1.4.0. 'ie/d/j/simatic/v1/{providerAppInstanceId}/dp/sr/req/{dpConnectionNamePath}{dpCollectionNamePath}': description: | Datapoint values of SIMATIC JSON payload format for single read requests. Added in version 1.4.0. 'ie/d/j/simatic/v1/{providerAppInstanceId}/dp/sr/resp/{dpConnectionNamePath}{dpCollectionNamePath}': description: | Datapoint values of SIMATIC JSON payload format for single read responses Added in version 1.4.0. 'ie/s/j/simatic/v1/{providerAppInstanceId}/status': description: | Status of the connector and its connections. Added in version 1.2. 'ie/c/j/simatic/v1/updaterequest': description: | Request to re-send the last values. Added in version 1.3.0. ``` To find the content definition of the messages, follow the `$ref` to the `message`. The best way is to use a yaml capable editor which allow navigating along the `ref`. We typically use Visual Studio code with the 'YAML' extension from Red Hat. Here the `message` ref for the metadata: ```yaml message: $ref: '#/components/messages/dpMetadataSimaticV1' ``` and here the definition of its message `dpMetadataSimaticV1` : ```yaml dpMetadataSimaticV1: description: | Allowed to be 'null' in version 1.4.0 for the shutdown or uninstallation of a connector. name: dpMetadataSimaticV1 contentType: application/json payload: oneOf: - type: "null" - $ref: "#/components/schemas/dpMetadataSimaticV1Payload" ``` There might be more than one payload message type allowed here. Metadata, status and all other topics which use retain=true allow an empty message (type null) for removal of that message from the broker in shutdown scenarios. The single-read and the write have a success and global error message: ```yaml payload: oneOf: - $ref: "#/components/schemas/subSingleReadResponseDpValueSimaticV1Payload" - $ref: "#/components/schemas/subSingleReadResponseGlobalErrorDpValueSimaticV1Payload" ``` In `dpMetadataSimaticV1Payload` you can see what properties are allowed and which are mandatory (check the `required` array). Most important are the description fields which give additional information (omitted here for brevity): ```yaml dpMetadataSimaticV1Payload: type: object additionalProperties: false required: - connections properties: seq: $ref: "#/components/schemas/sequenceNumber" hashVersion: $ref: "#/components/schemas/metadataHashVersion" connections: type: array items: $ref: "#/components/schemas/dpMetaConnectionV1" applicationName: type: string pattern: '^.{1,60}$' statustopic: type: string pattern: '^ie/s/j/simatic/v1/[^/]+/status$' examples: - "ie/s/j/simatic/v1/s7c1/status" ``` ### Metadata The starting point is always to subscribe to the MetaData to get the mapping from the DataPoint names to their IDs and also to get the needed topics to access the values or the status of a connector. You can subscribe to `ie/m/j/simatic/v1/#` to get all MetaData of all connectors and also use that to find the active connectors. ### Cyclic reading of Datapoint Values The topics under `ie/d/j/simatic/v1/{providerAppInstanceId}/dp/r` contain the cyclically updated values. There are two different update modes: - `CyclicOnChange`: Only the Datapoint values which have changed are contained in a message - `CyclicOnContinuous`: All Datapoint values are contained in all published messages Some connectors allow to configure the mode (e.g. per connection), others work in a fixed mode. To find out which topic you need to subscribe to get the values of a specific Datapoint, check out the Metadata. ### Re-publish the last published values of all Datapoints (update-request) When a connector uses the update mode `CyclicOnChange`, you typically have the issue that you missed the first message when a connector started up and your client will miss some slow-changing and constant values. To get the initial state for such a subscription, you can send an update-request message. This causes a re-publish of the last value of all subscribed DataPoints. The value will not be re-read from the device, instead the latest cached value together with the original timestamp and quality will be re-published under the original topic. That update message contains the property `"isResponseToUpdateRequest":true` so that a client can distinguish the update from a normal cyclic message. Normally you would ignore all responses that you didn't ask for, e.g. only accept the first one and after that ignore all following ones. The response message will be received by all clients so you should be careful only to request it when you really need it, that means typically after you subscribe to a new topic. **Don't call this cyclically because that could overload your system!** ### Single reading of Datapoint values "Single-read" is the term we used to distinguish from the cyclic read which is more a subscription. It is a simple request-reply which goes directly to the connector and reads the datapoint value from there. It doesn't use the value which might be cached in the Databus Gateway. There are DataPoints where it makes no sense to read them cyclically, e.g. recipe data or setpoints. For such use cases you can configure the DataPoint in the connector, but don't set the "Publish to Databus" and the "acquisition cycle" in the configuration. This will cause the DataPoint to be put into the `nonCyclicDataPoints` section of the MetaData. There you can find its ID and datatype. The ID can be used to do a single-read or a write. You can also do a single-read of DataPoints which are subscribed. You can find the request and response topics in the Metadata, here an example: - request: "singleReadRequestTopic" : "ie/d/j/simatic/v1/mbtcp1/dp/sr/req/s7" - response: "singleReadResponseTopic" : "ie/d/j/simatic/v1/mbtcp1/dp/sr/resp/s7" #### Example Command Get the Metadata and extract the read request (`singleReadRequestTopic`) and response (`singleReadResponseTopic`) topic from the metadata. ```bash mosquitto_sub -u edge -P edge -h ie-databus.proxy-redirect -t "ie/m/j/simatic/v1/mbtcp1/dp" -F '%j' | jq '.payload? | fromjson' ``` output from MetaData topic: ```json { "applicationName": "Modbus TCP Connector (Siemens AG)", "statustopic": "ie/s/j/simatic/v1/mbtcp1/status", "connections": [ { "name": "s7", "type": "cs-modbustcp", "dataPoints": [ { "name": "1000ms", "topic": "ie/d/j/simatic/v1/mbtcp1/dp/r/s7/1000ms", "pubTopic": "ie/d/j/simatic/v1/mbtcp1/dp/w/s7", "pubResponseTopic": "ie/d/j/simatic/v1/mbtcp1/dp/wr/resp/s7", "singleReadRequestTopic": "ie/d/j/simatic/v1/mbtcp1/dp/sr/req/s7", "singleReadResponseTopic": "ie/d/j/simatic/v1/mbtcp1/dp/sr/resp/s7", "publishType": "timeseries", "dataPointDefinitions": [ { "name": "Counter1", "id": "1", "dataType": "UInt", "accessMode": "rw", "acquisitionCycleInMs": 1000 }, { "name": "Counter2", "id": "2", "dataType": "UInt", "accessMode": "rw", "acquisitionCycleInMs": 1000 } ] } ], "nonCyclicDataPoints": { "pubTopic": "ie/d/j/simatic/v1/mbtcp1/dp/w/s7", "pubResponseTopic": "ie/d/j/simatic/v1/mbtcp1/dp/wr/resp/s7", "singleReadRequestTopic": "ie/d/j/simatic/v1/mbtcp1/dp/sr/req/s7", "singleReadResponseTopic": "ie/d/j/simatic/v1/mbtcp1/dp/sr/resp/s7", "dataPointDefinitions": [ { "name": "Counter1000", "id": "3", "dataType": "UInt", "accessMode": "rw" } ] } } ], "seq": 2, "hashVersion": 3165959619 } ``` Start a subscription to the response topic: ```bash singleReadResponseTopic="ie/d/j/simatic/v1/mbtcp1/dp/sr/resp/s7" mosquitto_sub -u edge -P edge -h ie-databus.proxy-redirect -t "$singleReadResponseTopic" -F '%j' | jq '.payload? | fromjson' ``` Execute the single-read command in another shell: ```bash singleReadRequestTopic="ie/d/j/simatic/v1/mbtcp1/dp/sr/req/s7" mosquitto_pub -u edge -P edge -h ie-databus.proxy-redirect -t "$singleReadRequestTopic" -m '{"requestId":"myclient-112371", "ids": ["1", "3"]}' ``` output from response topic when everything is ok: ```json { "requestId": "myclient-112371", "mdHashVer": 3165959619, "records": [ { "rseq": 10, "ts": "2025-11-04T15:19:53.4950688Z", "vals": [ { "id": "1", "qc": 3, "val": 16286 }, { "id": "3", "qc": 3, "val": 0 } ] } ] } ``` output from response topic when there is a global error because the request is malformed: ```json { "mdHashVer": 0, "globalError": { "resultCode": 11, "errorMessage": "Missing mandatory 'requestId' in 'Read Request'" } } ``` output from response topic when there is at least one datapoint specific errors (see `rejected`): ```json { "requestId": "myclient-112371", "mdHashVer": 3165959619, "records": [ { "rseq": 12, "ts": "2025-11-05T07:16:06.1819445Z", "vals": [ { "id": "3", "qc": 3, "val": 299 } ] } ] "rejected": [ { "id": "1", "resultCode": 6, "errorMessage": "ID is not a string" } ] } ``` ### Writing of Datapoint values Since API version V1.4.0 there is an optional response for the write message. To get that response, you need to provide the `requestId` property. Without it, there will be no response. This ensures compatibility with older versions and allows for faster performance when the response is not used anyway. There are DataPoints where it makes no sense to read them cyclically, but still want to write them. For such use cases you can configure the DataPoint in the connector, but don't set the "Publish to Databus" and the "acquisition cycle" in the configuration. This will cause the DataPoint to be put into the `nonCyclicDataPoints` section of the MetaData. There you can find its ID and datatype. The ID can be used to do a single-read or a write. You can find the request and response topics in the Metadata, here an example: - request: "pubTopic" : "ie/d/j/simatic/v1/mbtcp1/dp/w/s7" - response: "pubResponseTopic" : "ie/d/j/simatic/v1/mbtcp1/dp/wr/resp/s7" #### Example command Get the Metadata and extract the write request (`pubTopic`) and response (`pubResponseTopic`) topic from the metadata. ```bash mosquitto_sub -u edge -P edge -h ie-databus.proxy-redirect -t "ie/m/j/simatic/v1/mbtcp1/dp" -F '%j' | jq '.payload? | fromjson' ``` output from MetaData topic: ```json { "applicationName": "Modbus TCP Connector (Siemens AG)", "statustopic": "ie/s/j/simatic/v1/mbtcp1/status", "connections": [ { "name": "s7", "type": "cs-modbustcp", "dataPoints": [ { "name": "1000ms", "topic": "ie/d/j/simatic/v1/mbtcp1/dp/r/s7/1000ms", "pubTopic": "ie/d/j/simatic/v1/mbtcp1/dp/w/s7", "pubResponseTopic": "ie/d/j/simatic/v1/mbtcp1/dp/wr/resp/s7", "singleReadRequestTopic": "ie/d/j/simatic/v1/mbtcp1/dp/sr/req/s7", "singleReadResponseTopic": "ie/d/j/simatic/v1/mbtcp1/dp/sr/resp/s7", "publishType": "timeseries", "dataPointDefinitions": [ { "name": "Counter1", "id": "1", "dataType": "UInt", "accessMode": "rw", "acquisitionCycleInMs": 1000 }, { "name": "Counter2", "id": "2", "dataType": "UInt", "accessMode": "rw", "acquisitionCycleInMs": 1000 } ] } ], "nonCyclicDataPoints": { "pubTopic": "ie/d/j/simatic/v1/mbtcp1/dp/w/s7", "pubResponseTopic": "ie/d/j/simatic/v1/mbtcp1/dp/wr/resp/s7", "singleReadRequestTopic": "ie/d/j/simatic/v1/mbtcp1/dp/sr/req/s7", "singleReadResponseTopic": "ie/d/j/simatic/v1/mbtcp1/dp/sr/resp/s7", "dataPointDefinitions": [ { "name": "Counter1000", "id": "3", "dataType": "UInt", "accessMode": "rw" } ] } } ], "seq": 2, "hashVersion": 3165959619 } ``` Start a subscription to the response topic: ```bash pubResponseTopic="ie/d/j/simatic/v1/mbtcp1/dp/wr/resp/s7" mosquitto_sub -u edge -P edge -h ie-databus.proxy-redirect -t "$pubResponseTopic" -F '%j' | jq '.payload? | fromjson' ``` Execute the write command in another shell: ```bash pubTopic="ie/d/j/simatic/v1/mbtcp1/dp/w/s7" mosquitto_pub -u edge -P edge -h ie-databus.proxy-redirect -t "$pubTopic" -m '{"requestId":"myclient-90210", "vals": [{"id":"1", "val":123}, {"id":"3", "val":299}]}' ``` output from response topic when everything was fine: ```json { "requestId": "myclient-90210", "mdHashVer": 3165959619, "status": [ { "id": "1", "resultCode": 0 }, { "id": "3", "resultCode": 0 } ] } ``` output from response topic when there is a global error because the request is malformed: ```json { "requestId": "myclient-90210", "mdHashVer": 0, "globalError": { "resultCode": 10, "errorMessage": "Failed to read vals of 'Write Request'" } } ``` output from response topic when there is at least one datapoint specific errors (see `resultCode` not 0): ```json { "requestId": "myclient-90210", "mdHashVer": 3165959619, "status": [ { "id": "1", "resultCode": 6, "errorMessage": "ID is not a string" }, { "id": "3", "resultCode": 16, "errorMessage": "invalid argument for given datatype" } ] } ``` ### Status of a connector and its connections There is a status topic 'ie/s/j/simatic/v1/{providerAppInstanceId}/status' for each connector. It contains information about the connector itself which only tells if the connector is running. When the connector status is "good" it doesn't mean that the connection states are also good, those need to be evaluated separately. When displaying the status of a connector as one traffic light, you should merge the states of the connector and all connections in the following way: - connector state=="good" and all connection states=="good" => green - connector state=="good" and any connection state!="good" => yellow - connector state=="bad" => red Example status message: ```json { "seq": 5, "ts": "2025-11-04T15:09:17.7235090Z", "connector": { "status": "good", "reason": "CS-Connector up and running" }, "connections": [ { "name": "s7", "status": "good", "reason": "ok" } ] } ``` ## Quality Information There can be two types of quality information for each datapoint value: - qc: only 4 major states - qx: available since API V1.2 qx contains additional information compared with qc, for example sub-states for the qualities and some single bit flags. ### TL;DR Very Short Introduction - the `good` value for qc is 3 - the `good` value for the lower 8 bits of qx is 192 (hexadecimal 0xc0) or 128 (hexadecimal 0x80). That means you always need to mask the value with 255 (hexadecimal 0xff) before comparing because the bits 8 to 15 might be also set when the value is `good` - all other values mean `bad` or `uncertain` For more details how to evaluate or set qc/qx and the complete definition see below. ### qc This is a small subset of qx (actually the bits 7+6 of qx). qc can have one of the following numeric values: - 0 : BAD - The dp value is not useful - 1 : UNCERTAIN - The quality of the dp value is less than normal but the value may still be useful - 2 : GOOD (non-cascade) - The quality of the dp value is good - 3 : GOOD (cascade) - The quality of the dp value is good and may be used in control ### qx qx is an extension of qc which is 15 bits long instead of 2 bits. The Definition has been copied from WinCC UA. In the following, the overview of the qx structure: - High Byte: Industrial Edge Specific Information - Bit 15: Reserved - Bit 14...12: Flags - Bit 11...8: Extended sub-status - Low Byte: Quality code according to PROFIBUS PA / OPC DA - Bit 7...6: Quality - Bit 5...2: Sub-status - Bit 1...0: Limits #### Evaluation of qc/qx in a consumer App qx will be only added to a message, if it contains additional information to qc. That means, when any of the bits 14 to 8 or 5 to 0 is non-zero. As a consumer, if you would like to use the additional information of qx, you should do the following in your code to simplify the evaluation and to take care if the qx field is not there but just the qc field as fallback: ```C uint16_t qx; if(val.hasField("qx") { qx = val.getFieldValue("qx"); } else { // qx doesn’t exist, use qc as fallback and convert it to same structure as qx qx = val.getFieldValue("qc") << 6; } // work just with value of qx if(qx & 0x80) { // qx is either 'Good' or 'Good (cascade)' ... } ``` #### Quality Code The tables below reflect the quality, sub-status codes, and limit description of the quality code which is encoded in the low-byte of Industrial Edge quality attributes. ##### Quality | Value of bits 7..6 (decimal) | Quality | Description | | ---------------------------- | ------------------ | -------------------------------------------------------------------------------------------------------------------------- | | 0 | BAD | The value is not useful for reasons indicated by the sub-status. | | 1 | UNCERTAIN | The quality of the value is less than normal but the value may still be useful. The reason is indicated by the sub-status. | | 2 | GOOD (non-cascade) | The quality of the value is good. | | 3 | GOOD (cascade) | The quality of the value is good and may be used in control. | Only sub-status codes are listed that are assigned by the IE Connector. Sub-status codes assigned by the PLC and/or periphery only are not listed but reserved to provide peripheral status codes. ##### Sub-status for Quality "BAD" | Value of bits 5..2 (decimal) | "BAD" sub-status | Description | | ---------------------------- | ---------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 0 | Non-specific | There is no specific reason why the value is BAD. | | 1 | Configuration error | The value is not useful because of some inconsistency regarding the configuration. | | 2 | Not connected | The value is not reliable because the connection to the provider has been disconnected at consumer-side (e.g. a communication driver actively disconnects from a PLC on user request or by design). | | 4 | Sensor failure | The value is not useful because it cannot be converted, i.e. a value from the device (PLC) cannot be converted to the corresponding HMI tag. | | 5 | No communication, with last usable value | The value is not useful because the communication to the data source failed, however a last known value is available. | | 6 | No communication, no usable value | The value is not useful because the communication to the data source failed or has never been established since it was last out of service and a last known value is not available. | | 7 | Out of service | The value is not reliable because the provider side has been disabled or shut-down (e.g. a PLC is in stop mode, or a tag is disabled for maintenance purposes). | For further "BAD" sub-status codes assigned by PLC/periphery only, refer to PROFIBUS-PA. ##### Sub-status for Quality "UNCERTAIN" | Value of bits 5..2 (decimal) | "UNCERTAIN" sub-status | Description | | ---------------------------- | ---------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 0 | Non-specific | There is no specific reason why the value is UNCERTAIN. | | 1 | Last usable value | The connection to the data source is still established, however the data source stopped updating the value for some reason. | | 2 | Substitute value | A predefined value is used in case of an invalid value due to communication issues with the data source or a range violation. The reason for providing substitute values is configurable. | | 3 | Initial value | A predefined value intended for the startup of the HMI system (or a sub-ordinate device) is used while not being able to provide values from the data source. | | 5 | Range violation | The value lies outside the range defined by the min. and max. value. The limits define which direction (min. or max.) has been exceeded. | | 6 | Sub-normal | A value derived from multiple values has less than the required number of good sources. This includes data aggregation by means of data compression algorithms. | For further “UNCERTAIN” sub-status codes assigned by PLC/periphery only, refer to PROFIBUS-PA. ##### Sub-status for Quality "GOOD (cascade)" | Value of bits 5..2 (decimal) | "GOOD (cascade)" sub-status | Description | | ---------------------------- | --------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 0 | Non-specific | No error or special condition is associated with this value. | | 6 | Local override | The value has been overridden by the user or some logic in to continue operation. Typically, the input has been disconnected and a manually entered value has been 'forced', or a value has been corrected. | For further "GOOD" (non-cascade and cascade) sub-status codes assigned by PLC/periphery only, refer to PROFIBUS-PA. ##### Limits | Value of bits 1..0 (decimal) | "Limit" | Description | | ---------------------------- | -------------------- | ----------------------------------------------------------- | | 0 | Ok | Data quality unrelated to limits. | | 1 | Low limit violation | The value has exceeded its low limit. | | 2 | High limit violation | The value has exceeded its high limit. | | 3 | Constant | The value cannot move, no matter what the process is doing. | #### Industrial Edge specific Quality Information The tables below reflect the extended sub-status and flag description of the quality code which is Industrial Edge specific and encoded in the high-byte of Industrial Edge quality attributes. The intention of Industrial Edge specific extended sub-status is to describe the data quality more distinct than the sub-status according to PROFIBUS-PA. Nevertheless, to support 3rd party collaboration based on standards only, a best-effort sub-status is always set, even if a more distinct extended sub-status is present. ##### Extended sub-status "BAD" | Value of bits 11..8 (decimal) | Extended sub-status "BAD" | Description | | ----------------------------- | ------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 0 | Non-specific | No Industrial Edge specific extended bad sub-status is associated with this value. | | 1 | Aggregated value | The value has been calculated out of multiple values with less than the required number of good sources. This includes data aggregation by means of data compression algorithms. The corresponding sub-status is set to ‘non-specific’. | | 3 | Unusable value | A (logged) value has been identified to be incorrect but a respective correction value is not available. The corresponding sub-status is set to ‘non-specific’. | | 7 | Disabled | The provider of the value (e.g. logging tag for logged value) has been disabled and the previous value was BAD. The corresponding sub-status is taken from the last (previous) sub-status. | ##### Extended sub-status "UNCERTAIN" | Value of bits 11..8 (decimal) | Extended sub-status "UNCERTAIN" | Description | | ----------------------------- | ------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 0 | Non-specific | No Industrial Edge specific extended uncertain sub-status is associated with this value. | | 1 | Aggregated value | The value has been calculated out of multiple values with less than the required number of good sources to be GOOD as well as less than required number of bad sources to be BAD. This includes data aggregation by means of data compression algorithms. The corresponding sub-status is set to ‘sub-normal’. | | 7 | Disabled | The provider of the value (e.g. logging tag for logged value) has been disabled and the previous value was GOOD or UNCERTAIN. In case of GOOD, the corresponding sub- status is set to ‘last usable value’. In case of UNCERTAIN, the corresponding sub-status is taken from the last (previous) sub-status. | ##### Extended sub-status "GOOD (cascade)" | Value of bits 11..8 (decimal) | Extended sub-status "GOOD (cascade)" | Description | | ----------------------------- | ------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 0 | Non-specific | No Industrial Edge specific extended good sub-status is associated with this value. | | 1 | Aggregated value | The value has been calculated out of multiple (good) values. This includes data aggregation by means of data compression algorithms. The corresponding sub-status is set to ‘non-specific’. | | 2 | Manual input | A (logged) value has been created manually. The corresponding sub-status is set to ‘non-specific’. | | 3 | Corrected value | A (logged) value has been corrected. The corresponding sub-status is set to ‘non-specific’. | | 6 | Initial value | The local data source has been initialized with the configured initial value. The corresponding sub-status is set to ‘non-specific’. | ##### Extended sub-status "GOOD (non-cascade)" Please note that Industrial Edge doesn’t define extended sub-status for good (non-cascade). ##### Flags | Bit Number | Extended sub-status "GOOD (cascade)" | Description | | ---------- | ------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Bit 12 | Source quality | The data quality has been determined and assigned by external data source. | | Bit 13 | Source time | The data timestamp has been produced and assigned by external data source. | | Bit 14 | Time corrected | The data timestamp applied by external data source has been corrected by the system. Thus, Bit 13 “Source time” is not set. Time correction happens if the external timestamp is older than the timestamp of the last known value. | | Bit 15 | Reserved | Ignore when reading, set to 0 when creating, preserve when copying | ## Overview of Message Formats The following tables should give an overview about the different message types. ### Defined in Common Databus Payload API (asyncapi.yaml) These definitions are common for all connectors. #### Data Point Payload Format for read/subscribe | publishType | Message Type | Message Example in principle | Used by connector | | ---------------- | ------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------- | | single | Data Point Payload (deprecated since V1.2, no longer supported for new installations) | { seq, val, ts, qc } | SIMATIC S7 V1.1, Modbus TCP V1.0, Ethernet IP V1.0 | | bulk | Data Points Payload (subDpValueSimaticV1Payload) | { seq, (ts), vals[ { id, val, ts, qc, (qx) } ] } | SIMATIC S7 V1.1/V1.2, PROFINET IO V1.0.2 (no oversampling), Modbus TCP V1.0, Ethernet IP V1.0 | | timeseries | Data Points Time Series Payload (subDpValueSimaticV11TimeSeriesPayload) | { seq, records\[ { ts, rseq, vals[ { id, val, qc } ] } \] } | PROFINET IO V1.0.2 (with oversampling) | | binarytimeseries | Data Points Binary Payload (subDpValueSimaticBinaryMsgV1). Binary raw format with almost same content then “timeseries” format. | { seq, records\[ { ts, rseq, vals[ { id, val, qc } ] } \] } Currently only defined in user documentation of PROFINET IO Connector, not yet in asyncapi. | PROFINET IO V1.0.2 (with oversampling) | #### Data Point Payload Format for write/publish | publishType | Message Type | Message Example in principle | Used by connector | | ----------- | ------------------------------------------------------------------------------------- | ------------------------------------ | ------------------------------------------------------------------------ | | single | Data Point Payload (deprecated since V1.2, no longer supported for new installations) | { seq, vals{ id, val, ts, qc } } | SIMATIC S7 V1.1, Modbus TCP V1.0, Ethernet IP V1.0 | | bulk | Data Points Payload (pubDpValueSimaticV1Payload) | { seq, vals[ { id, val, ts, qc } ] } | SIMATIC S7 V1.1/V1.2, PROFINET IO 1.1, Modbus TCP V1.0, Ethernet IP V1.0 | #### PROFINET IO specific Payload Formats These messages are part of the Common Payload API but are defined in the separate file `asyncapi_profinet_io_conn.yaml`. They are specific for PROFINET and are only supported by the PROFINET IO Connector. Even in future, probably no other connector will implement them. | Message Type | Message Example in principle | Used by connector | | ------------------------------- | ----------------------------------------------------------------------------- | ----------------- | | Read PROFINET Records request | { (seq), reqs[ { laddr, index} ] } | PROFINET IO 1.1 | | Read PROFINET Records response | { (seq), rsps[ { (laddr), (index), (val), status, (errorDescription), ts} ] } | PROFINET IO 1.1 | | Write PROFINET Records request | { (seq), reqs[ { laddr, index, val} ] } | PROFINET IO 1.1 | | Write PROFINET Records response | { (seq), rsps[ { (laddr), (index), status, (errorDescription), ts} ] } | PROFINET IO 1.1 | ### Private Messages not defined in Common Databus Payload API These messages are local extensions of the connectors and are not part of the Common Databus API definition. That means, they have not been standardized in the Common Databus Payload API. The main reason is that the maturity level is not high enough to ensure that the definition will not change. In future versions of the API, they might be included but there may be some changes in the content or even the topic name. If a connector implements such private extensions, it probably must change the implementation when the message gets part of the Common Payload API with a changed content and also must define a transition strategy for clients which use the private extension. Client Apps should be aware that those private extensions have a higher chance of being changed than those of the Common Payload API messages. ### Other Payload Formats for read/subscribe | Message Type | Message Example in principle | Used by connector | | -------------------------------------------------------------------- | ----------------------------------------------------------------- | -------------------- | | Connection and Disconnection Status (not yet part of common payload) | { id, msg } | SIMATIC S7 V1.1/V1.2 | | Alarms | { seq, evs\[ { area, clsName, evTxt[ ], evTxtExt[ ], id, … } \] } | SIMATIC S7 V1.1/V1.2 | ## Scope of Datapoint Names The scope of a datapoint name is a connection. That means within a connection the name must be unique. ## Scope of the IDs The scope of a datapoint id is the connector instance. That means within a connector instance the id must be unique. ## Lifetime of the IDs and when do they change? When the Metadata is updated, the IDs can change. The metadata is only published on change (with `retain=true`), that means when you subscribe to it and receive an update you need to update your internal name to ID map. There is a field called `hashVersion` in the metadata which needs to match `mdHashVer` in several messages, e.g. for cyclic value updates and single reads and writes. When the values don't match, then your stored version of the metadata is outdated and needs to be updated. The mapping from `id` to `name` might have been changed in the new metadata. This is a synchronized way to know exactly which metadata version was used for which data message because the update of the metadata and data MQTT topics is asynchronously and maybe the notification order between Metadata message and Value message could be reversed (see also the sequence diagram below). ## Sequence Diagrams ### App Startup and Configuration Update ``` sequenceDiagram participant EdgeRuntime participant Connector participant Databus participant App EdgeRuntime ->> Databus : Start EdgeRuntime ->> Connector : Start Connector ->> Connector : Read configuration (hashVer=0x4567) Connector ->> Databus : Connect and Login Connector ->> Databus : Publish Metadata (retain=true, hashVer=0x4567) Connector ->> Databus : Publish Values(retain=false, seq=1, hashVer=0x4567) EdgeRuntime ->> App : Start App ->> Databus : Subscribe for Metadata and Values Databus ->> App : Notify Metadata (retain=true, hashVer=0x4567) App ->> App: Build Name <=> Id Map Connector ->> Databus : Publish values(retain=false, seq=2, hashVer=0x4567) Databus ->> App : Notify Values(retain=false, seq=2, hashVer=0x4567) Connector ->> Databus : Publish values(retain=false, seq=3, hashVer=0x4567) Databus ->> App : Notify Values(retain=false, seq=3, hashVer=0x4567) Connector ->> Connector : Detected Configuration Change Connector ->> Connector : Restart Connector ->> Connector : Read changed Configuration (hashVer=0x7825) alt normal case: Metadata notification arrives before first Value notification Connector ->> Databus : Publish Metadata (retain=true, hashVer=0x7825) (new hash) Databus ->> App : Notify Metadata (retain=true, hashVer=0x7825) App ->> App: Build Name <=> Id Map Connector ->> Databus : Publish Values(retain=false, seq=1, hashVer=07825) (seq restarts) Databus ->> App : Notify Values(retain=false, seq=1, hashVer=0x7825) else inversion of message notification order: Value before new Metadata Connector ->> Databus : Publish Metadata (retain=true, hashVer=0x7825) (new hash) Connector ->> Databus : Publish Values(retain=false, seq=1, hashVer=07825) (seq restarts) Databus ->> App : Notify Values(retain=false, seq=1, hashVer=0x7825) (requires updated metadata: ignore or put on hold) Databus ->> App : Notify Metadata (retain=true, hashVer=0x7825) App ->> App: Build Name <=> Id Map end ``` ## Conformance Testing / Validation ### Not-Conforming Connectors #### Legacy OPC UA connector The legacy OPC UA connector uses arrays packed into a string where the values are separated by pipe characters, e.g. “10 | 21 | 33”. This fails the validation because this is not an integer value (not even a quoted one). #### Legacy Ethernet IP and Modbus Connectors These connectors used Boolean values using “TRUE” and “FALSE” instead of 1 and 0. #### WinCC Unified MQTT adapter The WinCC Unified MQTT adapter uses IDs longer than the allowed 8 characters (e.g. “1.0.1000.2.0.0”). # Databus PROFINET IO Connector Payload Format AsyncApi [Download Async API Specification](asyncapi_profinet_io_conn.yaml) # Databus Common Payload Format AsyncApi [Download Async API Specification](asyncapi.yaml) # Databus Common Payload Format AsyncApi [Download Async API Specification](asyncapi.yaml) # Databus Common Payload Format AsyncApi [Download Async API Specification](asyncapi.yaml) # External Databus APIs | API | Version | | ---------- | ------------------------------------------------ | | Public API | [V1.0.0](v1.0.0/external_databus-api-1.0.0.html) | # External Databus API's The External Databus REST APIs allow you to configure users, topics certificate and bridges on External Databus instances running on multiple IE devices. ## OpenAPI Specification 1.0.0 [Download OpenAPI Specification](external_databus_V1.0.0_openapi.yaml) # Flow Creator Configurator APIs | API | Version | | ---------- | -------------------------------------------------------- | | Public API | [V2.0.0](v2.0.0/flowcreator-configurator-api-2.0.0.html) | | Public API | [V1.1.0](v1.1.0/flowcreator-configurator-api-1.1.0.html) | | Public API | [V1.0.0](v1.0.0/flowcreator-configurator-api-1.0.0.html) | # Flow Creator Configurator API The Flow Creator Configurator REST APIs allow you to configure flows, files, and nodes on Flow Creator instances running on multiple IE devices. ## OpenAPI Specification 1.0.0 [Download OpenAPI Specification](iefc-1.0.0-openapi.yaml) # Flow Creator Configurator API The Flow Creator Configurator REST APIs allow you to configure flows, files, and nodes on Flow Creator instances running on multiple IE devices. ## OpenAPI Specification 1.1.0 [Download OpenAPI Specification](iefc-1.1.0-openapi.yaml) # Flow Creator Configurator API The Flow Creator Configurator REST APIs allow you to configure flows, files, and nodes on Flow Creator instances running on multiple IE devices. ## OpenAPI Specification 2.0.0 [Download OpenAPI Specification](iefc-2.0.0-openapi.yaml) # Secure Storage This document provides a step-by-step guide for an application developer on how to use Secure Storage Services on the IED. Applications installed on the IED can use the Secure Storage Service to store secrets in an encrypted and secure manner, which cannot be accessed on the file system or by other applications. The secure storage service allows to add the data in the form of key-value pair, change the value for the existing pair, delete them, and retrieve them with their respective keys. ## Configurations - Application container must be connected to a proxy-redirect network or the application must use host network mode (not recommended) to make REST calls to the Secure Storage v2 APIs. - Host port 9443 is used by the system for this service and cannot be used for applications; if the application installed on the IED uses port 9443, then the application must be updated with a new port. ### How to use secure-storage V2.0.0 APIs with an app on IED The application must use SVID, which is SPIFFE Verifiable Identity Document, to use Secure Storage V2 APIs. They can be as follows: - X.509-SVID - The X.509-SVID is used to establish mutual TLS secured channels between the Industrial Edge Runtime Time (i.e. edge-iot-core) and the application container. - JWT-SVID - The JWT-SVID is used as an authorization token. These SVIDs can be retrieved from the SPIFFE Workload API via the mounted sock path `/var/run/devicemodel/edgedevice/edgeapiagent.sock`. SPIFFE supports libraries that allow interaction with the SPIFFE workload API for Go as `SPIFFE Go library` and Java as `SPIFFE Java library`. There also exists similar libraries in C, C++, Rust and Python. ## Procedure 1. Fetch the API server's SPIFFE ID from the container's `EDGE_SPIFFE_ID` environment variable and use it as follows - To configure mTLS client that will use X.509-SVID. for e.g. in Go, ```Go authorizer := tlsconfig.AuthorizeID(spiffeid.FromString(os.Getenv("EDGE_SPIFFE_ID"))) tlsConfig := tlsconfig.MTLSClientConfig(x509Source, x509Source, authorizer) ``` - As the audience of the JWT-SVID. for e.g. in Go, ```Go audience:= os.Getenv("EDGE_SPIFFEE_ID") svid, err := jwtSource.FetchJWTSVID(ctx, jwtsvid.Params{ Audience: audience,}) ``` For more details on language specific SPIFFE Library Usage refer [here](https://spiffe.io/docs/latest/deploying/libraries/). 1. Include the retrieved JWT-SVID in the `Authorization` header of the REST API call of secure storage with prefix `"JWT "`, e.g `JWT 551e145c-3a06-4d4b-99a3-3d0fd7185174` 1. The base API paths, depending on the configuration of the application container, are: - When the application connects to the proxy redirect API, the base path URL will be `https://edge-iot-core.proxy-redirect:8443/b.service/api/v2/secure-storage` - When the application connects, it either connects to the host network or has network mode as the host API base path URL will be `https://127.0.0.1:9443/b.service/api/v2/secure-storage` ## Secure Storage NFRs | Description | Value | Exceeding Values | | --------------------------------------------------------------------- | ----- | -------------------------------------------------------------------------------------------------- | | Maximum number of key-value pairs which can be stored per application | 50 | If more key-value pairs are being supplied to store, a proper error message is replied in response | | Maximum length of the key which can be stored | 256 B | If a larger key is supplied, a proper error message is replied in response | | Maximum length of the value which can be stored | 64 KB | If a larger value is supplied, a proper error message is replied in response | # IE Device APIs | API | Version | | ----------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | IE Device API | [V2.4.2](ied-api/v2.4.2/ied-api-2.4.2.html) [V2.4.1](ied-api/v2.4.1/ied-api-2.4.1.html) [V2.4.0](ied-api/v2.4.0/ied-api-2.4.0.html) [V2.1.0](ied-api/v2.1.0/ied-api-2.1.0.html) [V2.0.0](ied-api/v2.0.0/ied-api-2.0.0.html) [V1.0.0](ied-api/v1.0.0/ied-api-1.0.0.html) | | IE Device App Roles API | [V1.0.0](ied-app-roles-api/v1.0.0/ied-app-roles-api-1.0.0.html) | | IE Device OIDC API | [V1.0.0](ied-oidc-api/v1.0.0/ied-oidc-api-1.0.0.html) | | IE Device User API | [V1.0.0](ied-user/v1.0.0/ied-user-api-1.0.0.html) | | Secure Storage | [V2.0.0](secure-storage/v2.0.0/secure-storage-api-2.0.0.html) | | Service Registry | [V2.0.0](service_registry/RestAPIs.html) | # IE Device API 1.0.0 IE Device APIs offer the possibility to manage the lifecycle of the IE Device itself. This lifecycle includes the following: - Hard reset - Activation **Warning:** This API will be deprecated soon. Starting with IE Runtime v1.16, please use the [IE Device API 2.0](../v2.0.0/ied-api-2.0.0.html) specifications. ## OpenAPI Specification [Download OpenAPI Specification](ied-1.0.0-openapi.yaml) # IE Device API 2.0.0 IE Device APIs offer the possibility to manage the lifecycle of the IE Device itself, list its resources and publish notifications. These specifications are valid for IE Runtime version 1.13 to 1.22. ## OpenAPI Specification [Download OpenAPI Specification](ied-2.0.0-openapi.yaml) # Industrial Edge Device API 2.1.0 Industrial Edge Device APIs offer the possibility to manage the lifecycle of the Industrial Edge Device itself, list its resources, publish notifications and modify the hostname. These specifications are valid for IE Runtime version 1.23 and higher. ## OpenAPI Specification [Download OpenAPI Specification](ied-2.1.0-openapi.yaml) # Industrial Edge Device API 2.4.0 Industrial Edge Device APIs offer the possibility to manage the lifecycle of the Industrial Edge Device itself, list its resources, publish notifications and modify the hostname. These specifications are valid for IE Runtime version 1.25.0.7 and higher. ## OpenAPI Specification [Download OpenAPI Specification](ied-2.4.0-openapi.yaml) # Industrial Edge Device API 2.4.1 Industrial Edge Device APIs offer the possibility to manage the lifecycle of the Industrial Edge Device itself, list its resources, publish notifications and modify the hostname. These specifications are valid for IE Runtime version 1.25.0.7 and higher. ## OpenAPI Specification [Download OpenAPI Specification](ied-2.4.1-openapi.yaml) # Industrial Edge Device API 2.4.2 Industrial Edge Device APIs offer the possibility to manage the lifecycle of the Industrial Edge Device itself, list its resources, publish notifications and modify the hostname. These specifications are valid for IE Runtime version 1.25.0.7 and higher. ## OpenAPI Specification [Download OpenAPI Specification](ied-2.4.2-openapi.yaml) # Industrial Edge Device API 2.5.0 Industrial Edge Device APIs offer the possibility to manage the lifecycle of the Industrial Edge Device itself, list its resources, publish notifications and modify the hostname. These specifications are valid for IE Runtime version 1.25.0.7 and higher. ## OpenAPI Specification [Download OpenAPI Specification](ied-2.5.0-openapi.yaml) # IE Device App Role API The IED Auth API provides endpoints for managing system roles and user roles within the Industrial Edge Device. It includes operations to list, create, and delete system roles, as well as assign and remove roles for specific users. NOTICE This API is in the "Incubation" status. That means it is production-ready. However, it may be subject to changes, including backwards-incompatible changes. NOTICE User authorization for Industrial Edge Apps is available for Industrial Edge Runtime 1.23.0 or bigger. ## OpenAPI Specification [Download OpenAPI Specification](ied-app-roles-1.0.0-openapi.yaml) # IE Device OIDC Provider API IE Device OIDC provider APIs offer the possibility to manage OIDC provider. An OIDC provider is used to validate the identity of an existing device user over the OpenID Connect protocol. A configured OIDC provider is shown at the Login screen of the device as a possibility to login with. Besides the possibility to manage OIDC providers the API also includes necessary endpoints for the OIDC protocol. ## OpenAPI Specification [Download OpenAPI Specification](ied-oidc-1.0.0-openapi.yaml) # IE Device User API The IED User API provides endpoints for managing users within the Industrial Edge Device. It includes operations to list users. NOTICE This API is in the "Incubation" status. That means it is production-ready. However, it may be subject to changes, including backwards-incompatible changes. ## OpenAPI Specification [Download OpenAPI Specification](ied-user-1.0.0-openapi.yaml) # Secure Storage API 2.0.0 This application uses a secure storage system that manages data as key-value pairs. Each value is associated with a unique key, allowing precise identification and retrieval. - **Multiple key-value pairs:** An application can create and manage several key-value pairs, provided each key is distinct within that application. - **Unique application data:** Every application maintains its own separate set of keys and values, ensuring data isolation. ## Securing Communication To protect interactions between the IERT and the application container, the app can utilize SVIDs (Secure Volume Identifiers). - **Authentication:** Once retrieved, an SVID acts as an authentication token for accessing secure storage APIs, helping safeguard sensitive data. ## Data Storage Practices When saving data, the value is stored exactly as provided. - **User responsibility:** It is up to the application developer to ensure that any sensitive or confidential data is encrypted or otherwise protected before saving. The secure storage system does not automatically encrypt the values. ## Need Help Getting Started? For a step-by-step guide on securely storing and retrieving data, see [Guide: Secure Storage](../../../how-to/secure-storage.html). NOTICE These specifications are valid for IE Runtime version 1.19 and higher. ## OpenAPI Specification [Download Secure Storage v2 OpenAPI Specification](edgesecurity-2.0.0-openapi.yaml) # Service Registry APIs ## OpenAPI Specification [Download OpenAPI Specification](serviceregistry-2.0.0-openapi.yaml) # Industrial Edge Management APIs | API | Version | | ---------- | ------------------------------------------------- | | Admin API | [V1.0.0](api-admin/v1.0.0/api-admin-1-0-0.html) | | Portal API | [V1.9.0](api-portal/v1.9.0/api-portal-1-9-0.html) | # Admin API 1.0.0 In this section, you find the API for the admin user. ## OpenAPI Specification [Download OpenAPI Specification](admin-1.0.0-openapi.yaml) # Portal API 1.9.0 ## OpenAPI Specification [Download OpenAPI Specification](portal-1.9.0-openapi.yaml) # Industrial Edge Management V2 APIs `IEM Pro V2` introduces a new set of APIs built for improved performance and flexibility. Some `IEM Pro V1` APIs remain in use during the transition period. These legacy endpoints continue to support existing functionality and ensure backward compatibility. The following list outlines the `IEM Pro V1` APIs still active in `IEM Pro V2`, along with their purpose and expected deprecation timelines where applicable. - [IEM V1 API Overview](../iem/overview.html) # Portal API 2.0.0 ## OpenAPI Specification [Download OpenAPI Specification](portal-dev-v2-openapi.yaml) # Industrial Information Hub APIs | Component | API | | -------------- | --------------------------------------------------------------------------------------- | | IIH Essentials | [Public REST API](../../../industrial-information-hub/api/iih-essentials/overview.html) | | IIH Semantics | [OPC UA REST API](../../../industrial-information-hub/api/iih-semantics/overview.html) | # Vision Payload Specification | API | Version | | ---------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- | | Vision Payload Specification | [V1.0.0](v1.0.0/vision-payload-specification.html) [V1.1.0](v1.1.0/vision-payload-specification.html) [V1.1.2](v1.1.2/vision-payload-specification.html) | # Image Connector Applications Interface Specification ## *Version: 1.1.0* ## Contents - [Interface Definition of Camera Connectors](#interface-definition-of-camera-connectors) - [Version Handling and Compatibility](#version-handling-and-compatibility) - [General Definitions](#general-definitions) - [Topic Structure](#topic-structure) - [ZMQ](#zmq) - [MQTT](#mqtt) - [Device Identification](#device-identification) - [Messaging](#messaging) - [Vision Connector Image Format](#vision-connector-image-format) - [Camera Operation](#camera-operation) - [General Definitions](#general-definitions) - [Getting List of Supported Commands](#getting-list-of-supported-commands) - [Camera Discovery](#camera-discovery) - [Open Camera Connection](#open-camera-connection) - [Close Camera Connection](#close-camera-connection) - [Camera Parameter Configuration](#camera-parameter-configuration) - [Retrieving Parameter Names](#retrieving-parameter-names) - [Retrieving Parameter List](#retrieving-parameter-list) - [Setting Camera Parameters](#setting-camera-parameters) - [Camera Quick Settings](#camera-quick-settings) - [Camera User Settings](#camera-user-settings) - [Get Available Saved User Sets](#get-available-saved-user-sets) - [Load UserSet Configuration](#load-userset-configuration) - [Saving Configuration](#saving-configuration) - [Defining Default Configuration](#defining-default-configuration) - [Download Camera Configuration](#download-camera-configuration) - [Upload Camera Configuration](#upload-camera-configuration) - [Application Control](#application-control) - [Starting Streaming](#starting-streaming) - [Stop Streaming](#stop-streaming) - [Software Trigger Image Acquisition](#software-trigger-image-acquisition) - [Get Application Status](#get-application-status) - [Application Feedback](#application-feedback) - [Message Codes](#message-codes) - [Info message codes](#info-message-codes) - [Warning message codes](#warning-message-codes) - [Debug message codes](#debug-message-codes) - [Error message codes](#error-message-codes) - [Application Logging](#application-logging) - [Application Events](#application-events) - [Device Specific Events](#device-specific-events) - [Frame Warning Events](#frame-warning-events) ## Interface Definition of Camera Connectors This document outlines the northbound interface (indicating north in the automation pyramid) of a camera connector application and vision-driven applications. It covers the configuration of a connected camera and the retrieval/streaming of images from this camera for further processing. The southbound interface of the camera connector is defined by the connector provider, dependent on the technology used (GigEVision, GenICam, USBVision, etc.), and is therefore not part of this document. ## Version Handling and Compatibility This document is versioned in a \.\.\ schema. While patch and minor version increments must keep backwards compatibility with earlier versions of the standard, this may not be the case with an increment in the major version. The patch version must be incremented with every change in the document and is mainly used during incremental development of new features. Bigger changes in the document, like the addition of new commands or a large number of small additions, could be grouped to an increase in the minor version. Regarding backwards compatibility, the following rules apply: - Changes in commands are considered backwards compatible as long as using the old command still works and produces the same result. - Changes to responses are considered backward compatible if the interpretation of the response is still valid without the additional entries. - Adding MQTT and ZMQ topics is considered backwards compatible. - Adding new Message Codes is considered backwards compatible. Messages with such codes can be ignored in older versions. If a feature, command, or definition in this document is removed in a future standard version, it must be marked as being deprecated first and must stay present and usable for a defined amount of time until the feature may be removed. This would then cause a breaking change and therefore an increase in the major version. Proprietary commands, replies, and fields can be added according to vendors' needs but must be prefixed by "cstm\_" and must be compliant with the above-mentioned compatibility rules. Standard commands, replies, and fields must not start with this prefix. !!! info "NOTICE **Recommendation** Proprietary fields in proprietary commands do not need to add the prefix additionally. ## General Definitions To facilitate data transfer between the camera connector and vision-driven applications, the currently supported communication channels are [ZeroMQ (ZMQ)](https://zeromq.org/) and MQTT. For enhanced integration, such as with Siemens Industrial Edge, supplementary interfaces like REST API could also be considered. The connector application must possess two **bind** ZMQ ports and/or **MQTT** configurable connection parameters. These connections will be utilized for setting up the application, configuring the camera, defining image streaming options, and requesting images via software triggers. Images must adhere to the [Vision Connector Image Format](#vision-connector-image-format). All used ports and servers must be configurable in the application configuration file. The ZMQ ports (defaults: 5555 to REQ/REP and 5556 to PUB/SUB) must be documented in the application documentation. In the case of MQTT connections, the server, port, user, password and root topic should be configurable within the application. The connector should be designed to support one or multiple cameras. This support must be thoroughly detailed in the application documentation and should include a dedicated topic structure for each camera. The scope of this document is to define the interface protocols between vision-driven applications and camera connectivity applications. Communication specifics, camera protocols, camera setup, and application features are beyond the scope of this document. All definitions in this document, if not explicitly stated as optional, are mandatory for compliant implementations. Below is a schematic representation of the defined scope: ## Topic Structure ## ZMQ When using ZMQ connections, the expected structure outlined below: - To send commands, utilize single-frame messages sent to the REQ/REP port. - The image streaming topic on the PUB/SUB port: **device/{DeviceID}/stream** - For receiving human-readable application feedback, subscribe to topic on the PUB/SUB port (see also [Application Logging](#application-logging)): **app/logging** - For receiving events which should be handled by the subscriber, subscribe to topic on the PUB/SUB port (see also [Application Events](#application-events)): **app/events** ## MQTT For MQTT connections, the topic structure is as follows: - To send commands, publish messages to: **{MqttRoot}/app/request** - For receiving application command responses on: **{MqttRoot}/app/response** - Subscribe to the topic for receiving human-readable application feedback: **{mqttRoot}/app/logging** - For receiving events which should be handled by the subscriber: **{MqttRoot}/app/events** The message content should be formatted in JSON, employing the message field names as property names for each call, as described in the subsequent chapters of this document. Proprietary topics can be added according to vendors' needs but must be prefixed by "cstm\_". Standard topics must not start with this prefix. NOTICE **Recommendation** Proprietary (sub-)topics that fit into already standardized topics should be added to these as additional proprietary subtopics. If a new proprietary topic is introduced and is prefixed with "cstm\_", proprietary subtopics of it may not add the prefix additionally. Old topic structure from V1.0.0 is deprecated and will be removed in the next version. ## Device Identification To identify a device, the connector application should specify, in its [Camera Discovery](#camera-discovery) documentation, the allowed fields for identification. Typically, expected fields include the serial number (**SerialNumber** field), user identification (**UserDefinedName** field), IP address (for GigE devices, in the **IPAddress** field), and MAC address (for GigE devices, in the **MACAddress** field). While these fields are commonly used, they are not mandatory, may not be available, and the connector application may introduce additional or alternative identification fields. NOTICE The MAC address is typically delimited by ":" (e.g., 20:87:56:70:95:04). In the subsequent documentation, the term "DeviceID" refers to the identifying string that was used for a camera device upon opening it. This exact string will then be used for identifying the opened camera device while it is used and until it is closed again. Only after that the camera device can be opened and used again with a different DeviceID. The value of the DeviceID can be one of the fields returned during the device info (see more details in chapter [Open Camera Connection](#open-camera-connection)). Using a different DeviceID for a camera device while it is open, even if it refers to the same device, may lead to errors. Exceptions to this rule are allowed if documented by the application. DeviceIDs must not start with the proprietary suffix "cstm\_" to avoid conflicts. The following identifiers can't be used as a DeviceID: - "request" - "response" - "logging" - "app" - "device" ## Messaging The message exchange will be based on JSON messages. The subsequent sections will outline the minimum content for the message payload, where the fields are properties for the payload JSON, and any additional fields might be utilized as specified in the connector application documentation, in accordance with the vendor's requirements. Therefore, subscribers must make sure that their JSON parser accepts additional fields on top of the described ones. For ZMQ, the JSON string payload will be encoded into a UTF-8 single-frame message. In MQTT messages, the JSON payload serves as the content of the message to be sent. ## Vision Connector Image Format To standardize image transfer between camera streaming applications and image-driven applications, the following format shall be used for transferring images. The ZMQ message consists of a multipart message with each frame/part described below. This message format is prepared to support multiple stream cameras. ## Message Envelope | Frame | Field | Description | Type | | ------- | -------- | --------------------------------- | ---------------------------------------------------------------------------------------------- | | 1 | topic | Device identification. | Binary String (UTF-8) to identify the device in the following syntax: device/{DeviceID}/stream | | 2 | metadata | Metadata | Binary Metadata JSON (UTF-8) | | 3 | image | Image 1 | Binary | | 4 ... n | image | Image n – For multi-stream camera | Binary | ## Metadata JSON | Field | Optional | Type | Default | Description | | ------------ | -------- | ------------------------- | ------- | ---------------------------------------------------------------------------------------------------------------------------- | | version | No | String | 1 | Metadata format version | | count | No | Integer | 1 | Number of images in the message | | timestamp | No | String | -- | Date and Time string UTF-8 ISO8601. Application host datetime. | | customfields | Yes | String | -- | Custom field to be defined by the app creator and handled by the consumers. Might be a simple string or JSON string or else. | | detail | No | Image Detail Object Array | -- | Image details | ## Image Detail Object | Field | Optional | Type | Default | Description | | ----------- | -------------------------- | ------- | ------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | id | No | String | -- | Unique image identifier. Suggested mnemonic format like {camid}_{camstream}_{datetime} | | seq | No | Integer | -- | Sequential number for the camera image. Starting at 1, incremented by 1 on each frame grabbed. | | height | No / Yes (encoded formats) | Integer | -- | Image Height | | width | No / Yes (encoded formats) | Integer | -- | Image Width | | formatns | Yes | String | Genicam | Image format namespace. For GenICam pixel format use **Genicam** and **Compressed** for compressed formats. 2 | | format | No | String | -- | Image format 1 2 3 | | linepadding | Yes | Integer | 0 | Number of bytes added at the end of the byte stream to reach a multiple of 4 bytes. Eg. an image with 61 pixels, mono8, will have a line padding of 3, filling the stream to reach 64 bytes. | | timestamp | Yes | String5 | -- | Clock ticks informed by the camera.4 | | metadata6 | Yes | String | -- | Image additional metadata field to be defined by the app creator and handled by the consumers. Might be a simple string or JSON string or else. | 1 For cameras using a GenICam standardized pixel format, it is recommended to use the GenICam Naming Convention as described in section 4.35 [GenICam_PFNC_2_4.pdf](https://www.emva.org/wp-content/uploads/GenICam_PFNC_2_4.pdf). 2 Additional values can be supported once described in the application documentation. 3 Compressed formats shall be mentioned in the application documentation, being recommended in the fields corresponding to the file extension (e.g., "png", "jpg", "bmp"). 4 Timestamps provided by the cameras are Ticks count. This can be referencing the Unix epoch or the number of Ticks since the camera is on or count of Ticks informed/in sync with the PTP server. 5 A string might be used since, in the PTP standard, the Tick count is expected to be an 80-bit number. However, the current GenICam protocol returns a 64-bit integer. A string will be more reliable for future implementations. 6 Cameras capable and with PTP enabled must inform the PTP synchronization status in the metadata. E.g., ```json … "metadata": "{'ptpstatus': 'locked'}" … ``` ## Metadata Example ```json { "version": "1", "count": 2, "timestamp": "2014-01-09T13:35:34.000000000+0100", "customfields": "{'xfield1':1,'xfield2':'abc'}", "detail":[ { "id":"mycamera_0_2014-01-09T13:35:34.000000000+0100", "seq": 3, "height": 60, "width": 80, "format": "png", "formatns": "Compressed", "timestamp": "123456", "metadata": "{'batchid':'batch1','stream':'lowdef'}" }, { "id":"mycamera_1_2014-01-09T13:35:34.000000000+0100", "seq": 3, "height": 852, "width": 1280, "formatns": "Genicam", "format": "BGR8", "timestamp": "123456", "linepadding": 0, "metadata": "{'batchid':'batch1','stream':'raw'}" } ] } ``` ## Camera Operation ## General Definitions Camera Operations are handled by sending JSON strings via MQTT or ZMQ according to the [Topic Structure](#topic-structure). All operations have to be implemented by the connector application as long as the operation is not marked as "optional". Proprietary operations and fields can be added according to vendors needs but must be prefixed by "cstm\_". Operations and fields defined in this standard must not start with this prefix. If a requested command or argument isn't supported by the vision connector (e.g., the command in the given Action field of the command is unknown) then the vision connector must reply with the return code 0x03000003 (see also [Error message codes](#error-message-codes)). ## Getting List of Supported Commands To get the full list of commands that are currently supported by the vision connector, the client can send following command. ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | ------ | ------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *GetSupportedCommands* | ```json { "TransactionID": "123456", "Action": "GetSupportedCommands" } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | -------------------- | -------- | ---------------- | ------------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | | SupportedCommandList | No | Array of Strings | Array of commands (the possible values for the Action field). | ```json { "TransactionID": "123456", "ReturnCode": 0, "Message": "Supported commands retrieved.", "SupportedCommandList": [ "EnumerateDevices", "GetStatus", "OpenDevice", "CloseDevice", "StartStreaming", "StopStreaming", "GetParameterNames", "GetParameters", "SetParameters", "SetQuickParameters", "SoftwareTrigger", "GetUserSets", "LoadUserSet", "SaveUserSet", "SetDefaultUserSet", "SetDeviceConfig", "GetDeviceConfig", "GetSupportedCommands" ] } ``` ## Camera Discovery As the initial step to establish communication with a camera, it is necessary to check the available cameras on the connector. To achieve this, the connector shall be capable of discovering cameras (e.g., GigE cameras) and listing them for the user. ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | ------ | --------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *EnumerateDevices* | ```json { "TransactionID": "123456", "Action": "EnumerateDevices" } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | ------------- | -------- | ----------------------- | ------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | | DeviceList | No | Array of Camera Devices | Array of camera devices. | ## Camera Object | Field | Optional | Type | Description | | ----------------------------------- | -------- | ------ | ------------------------------------------------------------------------------------------------- | | SerialNumber1 | Yes | String | Camera identification | | UserDefinedName1 | Yes | String | Camera User-defined name | | ModelName | No | String | Camera model | | VendorName | No | String | Camera vendor | | Interface | No | String | Camera interface *U3V* (USB3 Vision), *CXP* (CoaXPress), *GEV* (GigEVision), *CamEmu* (Emulation) | | IPAddress | Yes | String | Camera IP address | | MACAddress | Yes | String | Camera MAC address | | {additional identification fields}1 | Yes | String | Additional device identification offered by the connector application. | 1 Device identification, must be informed on the documentation if they can be used as device identification or if there is additional fields that can be used. ```json { "TransactionID": "123456", "ReturnCode": 0, "Message": "Cameras found.", "DeviceList": [ { "SerialNumber": "548451887", "UserDefinedName": "MyCam1", "ModelName": "Camera Model 1234", "VendorName": "The camera Factory", "Interface": "GEV", "IPAddress": "192.168.0.1", "MACAddress": "00:1A:2B:3C:4D:5E" } ] } ``` ## Open Camera Connection Usually, GenICam cameras demand the connection to be open before starting to set parameters and image streaming. This connection is an exclusive access, meaning that no other app can access the camera and change parameters or stream images. A camera can be opened by specifying an identification string in the DeviceID field. The connector will then try to find a matching camera by searching for this identification string amongst the infos of the available cameras. The order in which it searches is the following: 1. MACAddress 1. SerialNumber 1. IPAddress 1. UserDefinedName 1. ModelName 1. VendorName 1. Interface Example: An identification string is provided in the DeviceID field. Then the connector will first try to find a camera with a matching MACAddress. If none was found it will try to find a camera with a matching SerialNumber. It will continue with the other criteria until a matching camera can be found and try to open this camera. If multiple cameras are found matching the same info then the connector will try to open the first one. It is up to the vision connector which camera is the first one and may even be randomly chosen. If a camera was opened using a certain identification string then this identification string must be used in all DeviceID fields of all requests regarding this camera until it is closed again. A camera can't be opened with multiple identification strings at the same time. ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | ------ | --------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *OpenDevice* | | DeviceID | No | String | Device identification. | ```json { "TransactionID": "1232156", "Action": "OpenDevice", "DeviceID": "548451887" } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | ------------- | -------- | ------- | ------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | ```json { "TransactionID": "123456", "ReturnCode": 0, "Message": "Camera open." } ``` ## Close Camera Connection To close the camera connection, releasing the exclusive access and stopping any running image streaming, releasing the camera. It is *not* considered an error if no camera with the specified DeviceID is open or the camera has already been closed. However, an respective message is provided in the response. ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | ------ | ---------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *CloseDevice* | | DeviceID | No | String | Device identification. | ```json { "TransactionID": "1232116", "Action": "CloseDevice", "DeviceID": "548451887" } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | ------------- | -------- | ------- | ------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | ```json { "TransactionID": "1232116", "ReturnCode": 0, "Message": "Camera closed." } ``` ## Camera Parameter Configuration ## Retrieving Parameter Names A list of all parameter names is provided when requested by the application user by sending the following message. If namespaces are provided in the request then the parameter names in the response must be prefixed with the according namespace (see example below). Otherwise, no prefix will be added in the response. ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | --------------- | ---------------------------------------------------------------------------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *GetParameterNames* | | DeviceID | No | String | Device identification. | | NamespaceList | Yes | Array of String | Namespaces of the parameter lists to query the parameter names from (defaults to camera parameter list if omitted). Example: | ```json { "TransactionID": "2345645", "Action": "GetParameterNames", "DeviceID": "548451887", "NamespaceList": [ "@CameraDevice", "@CameraInstance" ] } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | ----------------- | -------- | --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | | ParameterNameList | No | Array of String | String array containing the full parameters names (including the namespace prefixes for the node map if namespaces were provided in the request) | ```json { "TransactionID": "2345645", "ReturnCode": 0, "Message": "Parameter names retrieved successfully", "ParameterNameList": [ "@CameraDevice/Width", "@CameraDevice/Height", "@CameraDevice/PixelFormat", "@CameraInstance/BufferHandlingMode", ... ] } ``` ## Retrieving Parameter List A list of parameters is provided when requested by the application user. To request the parameter list, the user must send the following message. ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | --------------- | ------------------------------------------ | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *GetParameters* | | DeviceID | No | String | Device identification. | | ParameterList | No | Array of String | The list of parameters that shall be read. | ```json { "TransactionID": "2345645", "Action": "GetParameters", "DeviceID": "548451887", "ParameterList": [ "Width", "Height" ] } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | ------------- | -------- | ---------------------- | -------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | | ParameterList | No | Parameter Object Array | Parameter Object Array containing the parameters details | ## Parameter Object | Field | Optional | Type | Description | | -------------- | -------- | --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Name | No | String | Unique parameter name. | | DisplayName | Yes | String | Displayable name of the parameter for user interface. | | Type | No | String | Parameter value type. Possible values: *String*, *Integer*, *Float*, *Enumeration*, *Boolean*, *Command* | | Value | Yes | String/Int/Float/Bool | Current defined value. On enumerations, it is the string defined. Is not available if Type is Command or Readable is True. | | IntValue | Yes | Integer | On enumerations, it is the integer defined. | | Readable | No | Boolean | Flag indicating if the parameter can be read. Be aware that the readable flag depends of the current camera state. | | Writable | No | Boolean | Flag indicating if the parameter can be written. Be aware that the writable flag depends of the current camera state. | | Minimum | Yes | Int/Float | Minimum allowed value for numeric type parameters | | Maximum | Yes | Int/Float | Maximum allowed value for numeric type parameters | | Increment | Yes | Int/Float | Allowed increment for the parameter. | | Representation | Yes | String | Information on how this parameter should be represented in a user interface. This is only available for integer and float parameters. Possible values: *Linear*, *Logarithmic*, *PureNumber*, *Boolean*, *HexNumber*, *IPV4Address*, *MACAddress* | | Alias | Yes | String | Name of an alias parameter. An alias parameter represents the same functionality but in a different form (e.g., a float parameter could have an integer parameter as an alias). | | EnumEntries | Yes | Enum Entries Array | Array containing the entries available for the parameter. This is used by parameters that use enumerations. | ## Enum Entries | Field | Optional | Type | Description | | ----------- | -------- | ------- | ---------------------------------------------------------------- | | DisplayName | Yes | String | This represents the name of the property to the user | | Value | No | String | Parameter value that needs to be used to set the parameter value | | IntValue | No | Integer | Integer value of the enum entry | | Description | Yes | String | Enum entry description. | ```json { "TransactionID": "2345645", "ReturnCode": 0, "Message": "Parameters read successfully", "ParameterList": [ { "Name": "Width", "DisplayName": "Width", "Type": "Integer", "Value": 1024, "Writable": true, "Minimum": 1, "Maximum": 4096, "Increment": 1, "Representation": "Linear", "Alias": "ImageWidth" }, { "Name": "PixelFormat", "DisplayName": "Pixel Format", "Type": "Enumeration", "Value": "Mono8", "IntValue": 17301505, "Writable": true, "EnumEntries": [ { "DisplayName": "Mono 8", "Value": "Mono8", "IntValue": 17301505, "Description": "This enumeration value sets the pixel format to Mono 8." }, { "DisplayName": "Mono 10", "Value": "Mono10", "IntValue": 17825795, "Description": "This enumeration value sets the pixel format to Mono 10." }, { "DisplayName": "Mono 12", "Value": "Mono12", "IntValue": 17825797, "Description": "This enumeration value sets the pixel format to Mono 12." }, { "DisplayName": "Mono 16", "Value": "Mono16", "IntValue": 17825799, "Description": "This enumeration value sets the pixel format to Mono 16." } ] } ] } ``` ## Setting Camera Parameters To set the parameters, the user can send the message described in this chapter. It is possible to update a range of parameters; however, any error that may occur shall return the camera to the original state without applying any modification. Enumerations parameter types might be also defined using the human-readable value. For example, setting the pixel format using the value as Mono8, despite the enumeration integer value. For String and Integer parameters, the connector expects an exact match for the set value. For float parameters, a difference of no more than 1% is allowed due to floating-point comparison. If the provided value falls outside the expected range, the setting call will return an error. When using Command parameters, providing values has to result in an error message. The list of set values will be returned in the response message. ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | ---------------------- | ----------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *SetParameters* | | DeviceID | No | String | Device identification. | | ParameterList | No | Parameter Object Array | Object to encapsulate the parameters to update. | ## Parameter Object | Field | Optional | Type | Description | | ------ | -------------------------------------- | --------------------- | ------------------------------------------------------------------------------------------------------------------ | | Name | No | String | Unique Parameter Name2 | | Value1 | No / Yes (for command parameters only) | String/Int/Float/Bool | New value (the Value field must not be used for command parameters and must be used for all other parameter types) | 1On enumerations, it can be the integer value or can also be the symbolic name, refer to IntValue [Retrieving Parameter List](#retrieving-parameter-list). 2While it is recommended to use the unique parameter name, retrieved in [Retrieving Parameter List](#retrieving-parameter-list), applications may implement abbreviated parameter names as shortcuts to simplify the parameter settings or shortcuts to access multiple node maps. These shortcuts should be documented in the connector documentation. ```json { "TransactionID": "123567", "Action": "SetParameters", "DeviceID": "548451887", "ParameterList":[ {"Name": "Width", "Value": 800}, {"Name": "Height", "Value": 600}, {"Name": "PixelFormat", "Value": "Mono8"} //Setting using the human-readable string. This could be also the correspondent enum integer. ] } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | ------------- | -------- | ---------------------- | ------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | | ParameterList | No | Parameter Object Array | Object to encapsulate the parameters to update. | ## Parameter Object | Field | Optional | Type | Description | | ----- | -------- | --------------------- | ---------------------- | | Name | No | String | Unique Parameter Name2 | | Value | No | String/Int/Float/Bool | New value | ```json { "TransactionID": "123567", "ReturnCode": 0, "Message": "Parameters updated.", "ParameterList":[ {"Name": "Width", "Value": 800}, {"Name": "Height", "Value": 600}, {"Name": "PixelFormat", "Value": "Mono8"} ] } ``` ## Camera Quick Settings Additionally to the parameters settings, the quick setting feature provides a way to quickly define the most commonly used parameters. These parameters use common known names, and the connector is expected to map them internally to the camera correspondent parameter. ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | ---------------- | --------------------------------------------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *SetQuickParameters* | | DeviceID | No | String | Device identification. | | Width | Yes | Integer/String | Image width1 | | PixelFormat | Yes | Integer/String | Pixel format1,2 | | Height | Yes | Integer/String | Image Height1 | | Gain | Yes | Float/Int/String | Gain1. The string *Auto* can be used to activate automatic gain. | | ExposureTime | Yes | Float/Int/String | Exposure time in microseconds1. The string *Auto* can be used to activate automatic exposure. | 1 Camera dependent values, invalid values shall be informed on the returning error message. 2 GenICam standardized pixel format, according to the **GenICam Naming Convention** as described in section 4.35 [GenICam_PFNC_2_4.pdf](https://www.emva.org/wp-content/uploads/GenICam_PFNC_2_4.pdf). ```json { "TransactionID": "23456435", "Action": "SetQuickParameters", "DeviceID": "548451887", "Width": 800, "Height": 600, "Gain": "Auto", "PixelFormat": "Mono8", "ExposureTime": 234.5 } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | ------------- | -------- | ------- | ------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | ```json { "TransactionID": "23456435", "ReturnCode": 0, "Message": "Parameters updated." } ``` ## Camera User Settings The GenICam protocol allows the user to store parameter sets on the camera, and the connector must allow the user to save, load, and define the default user configuration to be used on camera restart. ## Get Available Saved User Sets Retrieve the available user sets on the camera. ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | ------ | ---------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *GetUserSets* | | DeviceID | No | String | Device identification. | ```json { "TransactionID": "1232116", "Action": "GetUserSets", "DeviceID": "12323454754" } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | ------------- | -------- | --------------------- | ------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | | UserSetList | No | User Set Object Array | Object containing the available parameter settings | ## User Set Object | Field | Optional | Type | Description | | ----------- | -------- | ------- | --------------------------------------------------------------- | | DisplayName | Yes | String | This represents the name of the user set to the user | | Value | No | String | Parameter value that needs to be used to set the user set value | | IntValue | No | Integer | Integer value of the user set | | Description | Yes | String | User set description. | ```json { "TransactionID": "12323454754", "ReturnCode": 0, "Message": "User set retrieved.", "UserSetList": [ { "DisplayName": "Default User Set", "Value": "Default", "IntValue": 0, "Description": "The default factory set can be loaded." }, { "DisplayName": "User Set 1", "Value": "UserSet1", "IntValue": 1, "Description": "User set 1 can be saved, loaded, or configured." }, { "DisplayName": "User Set 2", "Value": "UserSet2", "IntValue": 2, "Description": "User set 2 can be saved, loaded, or configured." }, { "DisplayName": "User Set 3", "Value": "UserSet3", "IntValue": 3, "Description": "User set 3 can be saved, loaded, or configured." } ] } ``` ## Load UserSet Configuration To load a configured UserSet on the camera, the user can send the *LoadUserSet* message. ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | ---------- | ------------------------------------------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *LoadUserSet* | | DeviceID | No | String | Device identification. | | UserSet | No | Int/String | String according with the user setting enumeration or integer when using the integer value. | ```json { "TransactionID": "23423423563456", "Action": "LoadUserSet", "DeviceID": "548451887", "UserSet": "UserSet2" } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | ------------- | -------- | ------- | ------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | ```json { "TransactionID": "1232116", "ReturnCode": 0, "Message": "User set loaded." } ``` ## Saving Configuration After the camera is parametrized, the parameter set can be saved on the camera on user sets. To perform the storage, the user can send the *SaveUserSet* message. ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | ---------- | ------------------------------------------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *SaveUserSet* | | DeviceID | No | String | Device identification. | | UserSet | No | Int/String | String according with the user setting enumeration or integer when using the integer value. | ```json { "TransactionID": "489494", "Action": "SaveUserSet", "DeviceID": "548451842342387", "UserSet": "UserSet2" } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | ------------- | -------- | ------- | ------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | ```json { "TransactionID": "489494", "ReturnCode": 0, "Message": "User set updated." } ``` ## Defining Default Configuration To define the camera default parameter set on the camera to be loaded in case of restart, the user can send the *SetDefaultUserSet* message. ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | ---------- | ------------------------------------------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *SetDefaultUserSet* | | DeviceID | No | String | Device identification. | | UserSet | No | Int/String | String according with the user setting enumeration or integer when using the integer value. | ```json { "TransactionID": "1232116", "Action": "SetDefaultUserSet", "DeviceID": "548451887", "UserSet": "UserSet2" } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | ------------- | -------- | ------- | ------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | ```json { "TransactionID": "1232116", "ReturnCode": 0, "Message": "Default user set defined." } ``` ## Download Camera Configuration The current camera setup might be downloaded to back up and further usages. The file format might be defined in the implementation, it is not meant to be edited externally. ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | ------ | -------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *GetDeviceConfig* | | DeviceID | No | String | Device identification. | ```json { "TransactionID": "23423456", "Action": "GetDeviceConfig", "DeviceID": "548451887" } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | ------------- | -------- | ------- | ------------------------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | | FileContent | No | String | Base64 string of the configuration file. Containing the mime type prefix. | ```json { "TransactionID": "23423456", "ReturnCode": 0, "Message": "Success", "FileContent": "data:text/plain;base64,IyB7MDVEOEMyOTQtRjI5NS00ZGZiLTlEMDEtMDk2QkQwNDA0OUY0fQojIEdlbkFwaSBwZXJzaXN0ZW5jZSBmaWxlICh2ZXJzaW9uIDMuMS4wKQojIERldmljZSA9IEJhc2xlcjo6QmFzbGVyQ2FtRW11IC0tIEJhc2xlciBDYW0gRW11IGludGVyZmFjZSAtLSBEZXZpY2UgdmVyc2lvbiA9IDEuMC4xIC0tIFByb2R1Y3QgR1VJRCA9IEI5N0RFNTQ5LUNCOUEtNDRmOC05MzdELTY0QzQ4QkRFQ0ZBMyAtLSBQcm9kdWN0IHZlcnNpb24gR1VJRCA9IEEyRENERTE0LTczRTQtNDA4YS1BRkE4LTgxODUyRUNCNUQ5QQpHYWluCTAuMDAwZSswMApXaWR0aAkxMDI0CkhlaWdodAkxMDQwClBpeGVsRm9ybWF0CU1vbm84CkV4cG9zdXJlVGltZQkxMDAwMC4wCg==" } ``` ## Upload Camera Configuration The stored file in section [Download Camera Configuration](#download-camera-configuration) can be provided to the camera restoring the setup. File integrity and validations are recommended and shall be implemented by the connector application. ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | ------ | ------------------------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *SetDeviceConfig* | | DeviceID | No | String | Device identification. | | FileContent | No | String | Base64 string of the configuration file. Containing the mime type prefix. | ```json { "TransactionID": "234423465465", "Action": "SetDeviceConfig", "DeviceID": "548451887", "FileContent": "data:text/plain;base64,IyB7MDVEOEMyOTQtRjI5NS00ZGZiLTlEMDEtMDk2QkQwNDA0OUY0fQojIEdlbkFwaSBwZXJzaXN0ZW5jZSBmaWxlICh2ZXJzaW9uIDMuMS4wKQojIERldmljZSA9IEJhc2xlcjo6QmFzbGVyQ2FtRW11IC0tIEJhc2xlciBDYW0gRW11IGludGVyZmFjZSAtLSBEZXZpY2UgdmVyc2lvbiA9IDEuMC4xIC0tIFByb2R1Y3QgR1VJRCA9IEI5N0RFNTQ5LUNCOUEtNDRmOC05MzdELTY0QzQ4QkRFQ0ZBMyAtLSBQcm9kdWN0IHZlcnNpb24gR1VJRCA9IEEyRENERTE0LTczRTQtNDA4YS1BRkE4LTgxODUyRUNCNUQ5QQpHYWluCTAuMDAwZSswMApXaWR0aAkxMDI0CkhlaWdodAkxMDQwClBpeGVsRm9ybWF0CU1vbm84CkV4cG9zdXJlVGltZQkxMDAwMC4wCg==" } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | ------------- | -------- | ------- | ------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | ```json { "TransactionID": "234423465465", "ReturnCode": 0, "Message": "Configuration loaded." } ``` ## Application Control The connector application shall allow the user to start and stop the image streaming. The control should not affect any camera property, including the trigger definitions. All camera parameter setups must be done through the parameter setup section. ## Starting Streaming This command is used to initiate image streaming for a specific camera. Upon activation, continuous streaming will publish images to the streaming topic (refer to [Topic Structure](#topic-structure)). For triggered grabbing, the application will send images based on trigger activations configured in the parameters section. If the optional FrameCount parameter is specified, the camera will stream the specified number of frames. It will automatically stop streaming when the frame count is reached (in this case, an event must also be sent to indicate that streaming has stopped, as described here [Stop Streaming](#stop-streaming)). If no value is specified for the FrameCount parameter, the camera will continue to stream until the user stops streaming. If a value less than 1 is specified for the FrameCount parameter, an error message must be sent. ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | ------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *StartStreaming* | | DeviceID | No | String | Device identification. | | FrameCount | Yes | Integer | Number of frames to be streamed. If no frame count is provided, the camera will stream continuously until the user stops the streaming. Values smaller than 1 are not valid and result in an error message | ```json { "TransactionID": "12341263412635465", "Action": "StartStreaming", "DeviceID": "548451887", "FrameCount": 100 } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | ------------- | -------- | ------- | ------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | ```json { "TransactionID": "12341263412635465", "ReturnCode": 0, "Message": "Stream started." } ``` ## Stop Streaming This command stops streaming images from the camera. It is *not* considered an error if the camera with the specified DeviceID is not currently streaming, or if streaming has already been stopped. However, an appropriate message should be returned in the response. When streaming is stopped (either automatically or via [Stop Streaming](#stop-streaming) command) the vision connector must notify this as an event using the message code 0x00000003 (see also [Info message codes](#info-message-codes) and [Device Specific Events](#device-specific-events)). ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | ------ | ------------------------------------------ | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *StopStreaming* | | DeviceID | No | String | Device identification. | ```json { "TransactionID": "32143242356465", "Action": "StopStreaming", "DeviceID": "548451887" } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | ------------- | -------- | ------- | ------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | ```json { "TransactionID": "32143242356465", "ReturnCode": 0, "Message": "Stream stopped." } ``` ## Software Trigger Image Acquisition This command is used when the camera is in software trigger operation. Upon execution, it triggers image acquisition as configured on the camera. Images will be published on the image streaming topic (refer to [Topic Structure](#topic-structure)). ## Message Envelope For ZMQ Message\*\* | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | ------ | -------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *SoftwareTrigger* | | DeviceID | No | String | Device identification. | ```json { "TransactionID": "2342546345", "Action": "SoftwareTrigger", "DeviceID": "548451887" } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | Response Object ______________________________________________________________________ | Field | Optional | Type | Description | | ------------- | -------- | ------- | ------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | ```json { "TransactionID": "2342546345", "ReturnCode": 0, "Message": "Trigger sent." } ``` ## Get Application Status Retrieves the connected camera list with status, streaming topic, application version, and implemented specification version. For compatibility purposes, both the legacy streaming topic (refer to the previous version of this standard) and the new streaming topic (refer to[ZMQ](#zmq)) are supported. It is recommended to return the new streaming topic in the GetStatus response. ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | ------ | -------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *GetStatus* | ```json { "TransactionID": "46543214635244", "Action": "GetStatus" } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | -------------------- | -------- | ---------------------- | ------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | | ApplicationName | Yes | String | Connector application name | | ApplicationVersion | No | String | Connector application version | | SpecificationVersion | No | String | Specification version of the connector | | DeviceList | No | Array of Camera Object | Array of camera objects | ## Camera Object | Field | Optional | Type | Description | | ----------------------------------- | -------- | ------ | ------------------------------------------------------------------------------------------------- | | DeviceID | No | String | Current camera identification | | StreamingTopic | No | String | Topic where the camera streams images | | Status | No | String | Camera status. Possible values: *Connected*, *Streaming*, *Error* | | SerialNumber1 | Yes | String | Camera identification | | UserDefinedName1 | Yes | String | Camera User-defined name | | ModelName1 | No | String | Camera model | | VendorName1 | No | String | Camera vendor | | Interface1 | No | String | Camera interface *U3V* (USB3 Vision), *CXP* (CoaXPress), *GEV* (GigEVision), *CamEmu* (Emulation) | | IPAddress1 | Yes | String | Camera IP address | | MACAddress | Yes | String | Camera MAC address | | {additional identification fields}1 | Yes | String | Additional device identification offered by the connector application. | 1 Device identification, must be informed on the documentation if they can be used as device identification or if there are additional fields that can be used. ```json { "TransactionID": "46543214635244", "ReturnCode": 0, "Message": "Success.", "ApplicationName": "Demo Vision Connector", "ApplicationVersion": "1.0.1", "SpecificationVersion": "1.1.0", "DeviceList": [ { "DeviceID": "548451887", "StreamingTopic": "device/548451887/stream", "Status": "Streaming", "SerialNumber": "548451887", "UserDefinedName": "MyCam1", "ModelName": "Camera Model 1234", "VendorName": "The camera Factory", "Interface": "GEV", "IPAddress": "192.168.0.1", "MACAddress": "00:1A:2B:3C:4D:5E" } ] } ``` ## Application Feedback ## Message Codes The following message codes are expected for the responses. There are standard message codes that are standardized by this document, and custom message codes that can be defined by each Vision Connector individually. Message codes are 32-bit integers that are encoded in the following way: | Bit offset (lsb \<< x) | Width (bits) | Description | | ---------------------- | ------------ | ------------------------------------------------------------------------------------------------------------------------------------- | | 0 | 20 | Index of the message code (starting at 0 for each log level) | | 20 | 3 | Reserved bits (must be set to 0) | | 23 | 1 | Must be set to 1 for custom message codes or 0 otherwise | | 24 | 2 | Log level. The following log levels exist at the moment: 0 = Info messages 1 = Warning messages 2 = Debug messages 3 = Error messages | | 26 | 6 | Reserved bits (must be set to 0) | ## Info message codes Value range for standard info messages: 0x00000000 ... 0x000FFFFF Value range for custom info messages: 0x00800000 ... 0x008FFFFF | Standard Message Codes | Meaning | | ---------------------- | ---------------------------------- | | 0x00000000 | Request was completed successfully | | 0x00000001 | Application is online | | 0x00000002 | Application is offline | | 0x00000003 | Streaming of a device is stopped | ## Warning message codes Value range for standard warning messages: 0x01000000 ... 0x010FFFFF Value range for custom warning messages: 0x01800000 ... 0x018FFFFF | Standard Message Codes | Meaning | | ---------------------- | -------------------------------------------- | | 0x01000000 | A request was only partially successful | | 0x01000001 | One or multiple corrupt frames were received | | 0x01000002 | One or multiple dropped frames were detected | ## Debug message codes Value range for standard debug messages: 0x02000000 ... 0x020FFFFF Value range for custom debug messages: 0x02800000 ... 0x028FFFFF ## Error message codes Value range for standard error messages: 0x03000000 ... 0x030FFFFF Value range for custom error messages: 0x03800000 ... 0x038FFFFF | Standard Message Codes | Meaning | | ---------------------- | -------------------------------------------------------- | | 0x03000000 | Error while parsing a request | | 0x03000001 | Error while handling a request | | 0x03000002 | An opened device was removed | | 0x03000003 | A feature (e.g., a command or argument) is not supported | ## Application Logging The application log is expected to be published on the logging topic, any general application log which is not a direct response to any given command, like application unhandled errors, camera error messages, and camera connectivity issues. The purpose of this log is to provide a human-readable application feedback that is not intended for interpretation by any software. ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | -------------------------------- | | 1 | topic | Logging topic | Binary String (UTF-8): *logging* | | 2 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | --------- | -------- | ------- | ------------------------------------------------------------------------------- | | Timestamp | Yes | String | Application host date and time on ISO8601 format, when the message was created. | | Code | Yes | Integer | Message code following [Message Codes](#message-codes). | | Level | No | String | Log level - *Debug*, *Info*, *Warning*, *Error* | | Message | No | String | Response message | ```json { "Timestamp":"2014-01-09T13:35:34.000000000+0100", "Code": 50331650, "Level":"Error", "Message":"Lost connection with myCam1" } ``` ## Application Events Application events are expected to be published on the event topic, any general application events that are not direct responses to a specific command, such as unhandled application errors, camera error messages, and camera connectivity problems. Unlike the application log, the purpose of these events is to provide notifications that can be interpreted by software. Certain message types (identified by the message code) may contain additional data. However, this is only required for the message codes listed below. The messages in the events are Vision Connector specific and must not be interpreted. ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | -------------------------------- | | 1 | topic | Logging topic | Binary String (UTF-8): *logging* | | 2 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | -------------- | --------------- | ---------------------- | ----------------------------------------------------------------------------------------------------------------------------- | | Timestamp | Yes | String | Application host date and time on ISO8601 format, when the message was created. | | Code | No | Integer | Message code following [Message Codes](#message-codes). | | Level | No | String | Log level - *Debug*, *Info*, *Warning*, *Error* | | Message | No | String | Response message | | AdditionalData | Depends on Code | Additional Data Object | Additional message code dependent data. AdditionalData does not exist unless explicitly specified for a certain message code. | ```json { "Timestamp":"2014-01-09T13:35:34.000000000+0100", "Code": 2, "Level":"Info", "Message":"The vision connector is offline now." } ``` ## Device Specific Events Device specific events must contain additional data that contains the entry "DeviceID", which will provide the device ID that this event happened on. List of all device specific events: - 0x00000003 | Streaming of a device is stopped - 0x03000002 | An opened device was removed ## Additional Data Object | Field | Optional | Type | Description | | -------- | -------- | ------ | ----------------------------- | | DeviceID | No | String | Current camera identification | Example event for a device connection loss: ```json { "Timestamp":"2014-01-09T13:35:34.000000000+0100", "Code": 50331650, "Level":"Error", "Message":"Lost connection with myCam1", "AdditionalData": { "DeviceID": "myCam1" } } ``` ## Frame Warning Events Frame warning events must contain additional data that contains the entry "DeviceID", which will provide the device ID that this event happened on. Furthermore, a "FrameCount" must be provided if more than 1 frame is affected. List of all frame warning events: - 0x01000001 | One or multiple corrupt frames were received - 0x01000002 | One or multiple dropped frames were received ## Additional Data Object | Field | Optional | Type | Description | | ---------- | -------- | ------- | ------------------------------------------------------------------------------------------------------ | | DeviceID | No | String | Current camera identification | | FrameCount | Yes | Integer | The number of images affected (if no frame count is provided then this is equal to a frame count of 1) | Example event for dropped frames: ```json { "Timestamp":"2014-01-09T13:35:34.000000000+0100", "Code": 16777218, "Level":"Warning", "Message":"3 dropped frames have been detected on myCam1", "AdditionalData": { "DeviceID": "myCam1", "FrameCount": 3 } } ``` # Image Connector Applications Interface Specification ## *Version: 1.1.0* ## Contents - [Interface Definition of Camera Connectors](#interface-definition-of-camera-connectors) - [Version Handling and Compatibility](#version-handling-and-compatibility) - [General Definitions](#general-definitions) - [Topic Structure](#topic-structure) - [ZMQ](#zmq) - [MQTT](#mqtt) - [Device Identification](#device-identification) - [Messaging](#messaging) - [Vision Connector Image Format](#vision-connector-image-format) - [Camera Operation](#camera-operation) - [General Definitions](#general-definitions) - [Getting List of Supported Commands](#getting-list-of-supported-commands) - [Camera Discovery](#camera-discovery) - [Open Camera Connection](#open-camera-connection) - [Close Camera Connection](#close-camera-connection) - [Camera Parameter Configuration](#camera-parameter-configuration) - [Retrieving Parameter Names](#retrieving-parameter-names) - [Retrieving Parameter List](#retrieving-parameter-list) - [Setting Camera Parameters](#setting-camera-parameters) - [Camera Quick Settings](#camera-quick-settings) - [Camera User Settings](#camera-user-settings) - [Get Available Saved User Sets](#get-available-saved-user-sets) - [Load UserSet Configuration](#load-userset-configuration) - [Saving Configuration](#saving-configuration) - [Defining Default Configuration](#defining-default-configuration) - [Download Camera Configuration](#download-camera-configuration) - [Upload Camera Configuration](#upload-camera-configuration) - [Application Control](#application-control) - [Starting Streaming](#starting-streaming) - [Stop Streaming](#stop-streaming) - [Software Trigger Image Acquisition](#software-trigger-image-acquisition) - [Get Application Status](#get-application-status) - [Application Feedback](#application-feedback) - [Message Codes](#message-codes) - [Info message codes](#info-message-codes) - [Warning message codes](#warning-message-codes) - [Debug message codes](#debug-message-codes) - [Error message codes](#error-message-codes) - [Application Logging](#application-logging) - [Application Events](#application-events) - [Device Specific Events](#device-specific-events) - [Frame Warning Events](#frame-warning-events) ## Interface Definition of Camera Connectors This document outlines the northbound interface (indicating north in the automation pyramid) of a camera connector application and vision-driven applications. It covers the configuration of a connected camera and the retrieval/streaming of images from this camera for further processing. The southbound interface of the camera connector is defined by the connector provider, dependent on the technology used (GigEVision, GenICam, USBVision, etc.), and is therefore not part of this document. ## Version Handling and Compatibility This document is versioned in a \.\.\ schema. While patch and minor version increments must keep backwards compatibility with earlier versions of the standard, this may not be the case with an increment in the major version. The patch version must be incremented with every change in the document and is mainly used during incremental development of new features. Bigger changes in the document, like the addition of new commands or a large number of small additions, could be grouped to an increase in the minor version. Regarding backwards compatibility, the following rules apply: - Changes in commands are considered backwards compatible as long as using the old command still works and produces the same result. - Changes to responses are considered backward compatible if the interpretation of the response is still valid without the additional entries. - Adding MQTT and ZMQ topics is considered backwards compatible. - Adding new Message Codes is considered backwards compatible. Messages with such codes can be ignored in older versions. If a feature, command, or definition in this document is removed in a future standard version, it must be marked as being deprecated first and must stay present and usable for a defined amount of time until the feature may be removed. This would then cause a breaking change and therefore an increase in the major version. Proprietary commands, replies, and fields can be added according to vendors' needs but must be prefixed by "cstm\_" and must be compliant with the above-mentioned compatibility rules. Standard commands, replies, and fields must not start with this prefix. !!! info "NOTICE **Recommendation** Proprietary fields in proprietary commands do not need to add the prefix additionally. ## General Definitions To facilitate data transfer between the camera connector and vision-driven applications, the currently supported communication channels are [ZeroMQ (ZMQ)](https://zeromq.org/) and MQTT. For enhanced integration, such as with Siemens Industrial Edge, supplementary interfaces like REST API could also be considered. The connector application must possess two **bind** ZMQ ports and/or **MQTT** configurable connection parameters. These connections will be utilized for setting up the application, configuring the camera, defining image streaming options, and requesting images via software triggers. Images must adhere to the [Vision Connector Image Format](#vision-connector-image-format). All used ports and servers must be configurable in the application configuration file. The ZMQ ports (defaults: 5555 to REQ/REP and 5556 to PUB/SUB) must be documented in the application documentation. In the case of MQTT connections, the server, port, user, password and root topic should be configurable within the application. The connector should be designed to support one or multiple cameras. This support must be thoroughly detailed in the application documentation and should include a dedicated topic structure for each camera. The scope of this document is to define the interface protocols between vision-driven applications and camera connectivity applications. Communication specifics, camera protocols, camera setup, and application features are beyond the scope of this document. All definitions in this document, if not explicitly stated as optional, are mandatory for compliant implementations. Below is a schematic representation of the defined scope: ## Topic Structure ## ZMQ When using ZMQ connections, the expected structure outlined below: - To send commands, utilize single-frame messages sent to the REQ/REP port. - The image streaming topic on the PUB/SUB port: **device/{DeviceID}/stream** - For receiving human-readable application feedback, subscribe to topic on the PUB/SUB port (see also [Application Logging](#application-logging)): **app/logging** - For receiving events which should be handled by the subscriber, subscribe to topic on the PUB/SUB port (see also [Application Events](#application-events)): **app/events** ## MQTT For MQTT connections, the topic structure is as follows: - To send commands, publish messages to: **{MqttRoot}/app/request** - For receiving application command responses on: **{MqttRoot}/app/response** - Subscribe to the topic for receiving human-readable application feedback: **{mqttRoot}/app/logging** - For receiving events which should be handled by the subscriber: **{MqttRoot}/app/events** The message content should be formatted in JSON, employing the message field names as property names for each call, as described in the subsequent chapters of this document. Proprietary topics can be added according to vendors' needs but must be prefixed by "cstm\_". Standard topics must not start with this prefix. NOTICE **Recommendation** Proprietary (sub-)topics that fit into already standardized topics should be added to these as additional proprietary subtopics. If a new proprietary topic is introduced and is prefixed with "cstm\_", proprietary subtopics of it may not add the prefix additionally. Old topic structure from V1.0.0 is deprecated and will be removed in the next version. ## Device Identification To identify a device, the connector application should specify, in its [Camera Discovery](#camera-discovery) documentation, the allowed fields for identification. Typically, expected fields include the serial number (**SerialNumber** field), user identification (**UserDefinedName** field), IP address (for GigE devices, in the **IPAddress** field), and MAC address (for GigE devices, in the **MACAddress** field). While these fields are commonly used, they are not mandatory, may not be available, and the connector application may introduce additional or alternative identification fields. NOTICE The MAC address is typically delimited by ":" (e.g., 20:87:56:70:95:04). In the subsequent documentation, the term "DeviceID" refers to the identifying string that was used for a camera device upon opening it. This exact string will then be used for identifying the opened camera device while it is used and until it is closed again. Only after that the camera device can be opened and used again with a different DeviceID. The value of the DeviceID can be one of the fields returned during the device info (see more details in chapter [Open Camera Connection](#open-camera-connection)). Using a different DeviceID for a camera device while it is open, even if it refers to the same device, may lead to errors. Exceptions to this rule are allowed if documented by the application. DeviceIDs must not start with the proprietary suffix "cstm\_" to avoid conflicts. The following identifiers can't be used as a DeviceID: - "request" - "response" - "logging" - "app" - "device" ## Messaging The message exchange will be based on JSON messages. The subsequent sections will outline the minimum content for the message payload, where the fields are properties for the payload JSON, and any additional fields might be utilized as specified in the connector application documentation, in accordance with the vendor's requirements. Therefore, subscribers must make sure that their JSON parser accepts additional fields on top of the described ones. For ZMQ, the JSON string payload will be encoded into a UTF-8 single-frame message. In MQTT messages, the JSON payload serves as the content of the message to be sent. ## Vision Connector Image Format To standardize image transfer between camera streaming applications and image-driven applications, the following format shall be used for transferring images. The ZMQ message consists of a multipart message with each frame/part described below. This message format is prepared to support multiple stream cameras. ## Message Envelope | Frame | Field | Description | Type | | ------- | -------- | --------------------------------- | ---------------------------------------------------------------------------------------------- | | 1 | topic | Device identification. | Binary String (UTF-8) to identify the device in the following syntax: device/{DeviceID}/stream | | 2 | metadata | Metadata | Binary Metadata JSON (UTF-8) | | 3 | image | Image 1 | Binary | | 4 ... n | image | Image n – For multi-stream camera | Binary | ## Metadata JSON | Field | Optional | Type | Default | Description | | ------------ | -------- | ------------------------- | ------- | ---------------------------------------------------------------------------------------------------------------------------- | | version | No | String | 1 | Metadata format version | | count | No | Integer | 1 | Number of images in the message | | timestamp | No | String | -- | Date and Time string UTF-8 ISO8601. Application host datetime. | | customfields | Yes | String | -- | Custom field to be defined by the app creator and handled by the consumers. Might be a simple string or JSON string or else. | | detail | No | Image Detail Object Array | -- | Image details | ## Image Detail Object | Field | Optional | Type | Default | Description | | ----------- | -------------------------- | ------- | ------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | id | No | String | -- | Unique image identifier. Suggested mnemonic format like {camid}_{camstream}_{datetime} | | seq | No | Integer | -- | Sequential number for the camera image. Starting at 1, incremented by 1 on each frame grabbed. | | height | No / Yes (encoded formats) | Integer | -- | Image Height | | width | No / Yes (encoded formats) | Integer | -- | Image Width | | formatns | Yes | String | Genicam | Image format namespace. For GenICam pixel format use **Genicam** and **Compressed** for compressed formats. 2 | | format | No | String | -- | Image format 1 2 3 | | linepadding | Yes | Integer | 0 | Number of bytes added at the end of the byte stream to reach a multiple of 4 bytes. Eg. an image with 61 pixels, mono8, will have a line padding of 3, filling the stream to reach 64 bytes. | | timestamp | Yes | String5 | -- | Clock ticks informed by the camera.4 | | metadata6 | Yes | String | -- | Image additional metadata field to be defined by the app creator and handled by the consumers. Might be a simple string or JSON string or else. | 1 For cameras using a GenICam standardized pixel format, it is recommended to use the GenICam Naming Convention as described in section 4.35 [GenICam_PFNC_2_4.pdf](https://www.emva.org/wp-content/uploads/GenICam_PFNC_2_4.pdf). 2 Additional values can be supported once described in the application documentation. 3 Compressed formats shall be mentioned in the application documentation, being recommended in the fields corresponding to the file extension (e.g., "png", "jpg", "bmp"). 4 Timestamps provided by the cameras are Ticks count. This can be referencing the Unix epoch or the number of Ticks since the camera is on or count of Ticks informed/in sync with the PTP server. 5 A string might be used since, in the PTP standard, the Tick count is expected to be an 80-bit number. However, the current GenICam protocol returns a 64-bit integer. A string will be more reliable for future implementations. 6 Cameras capable and with PTP enabled must inform the PTP synchronization status in the metadata. E.g., ```json … "metadata": "{'ptpstatus': 'locked'}" … ``` ## Metadata Example ```json { "version": "1", "count": 2, "timestamp": "2014-01-09T13:35:34.000000000+0100", "customfields": "{'xfield1':1,'xfield2':'abc'}", "detail":[ { "id":"mycamera_0_2014-01-09T13:35:34.000000000+0100", "seq": 3, "height": 60, "width": 80, "format": "png", "formatns": "Compressed", "timestamp": "123456", "metadata": "{'batchid':'batch1','stream':'lowdef'}" }, { "id":"mycamera_1_2014-01-09T13:35:34.000000000+0100", "seq": 3, "height": 852, "width": 1280, "formatns": "Genicam", "format": "BGR8", "timestamp": "123456", "linepadding": 0, "metadata": "{'batchid':'batch1','stream':'raw'}" } ] } ``` ## Camera Operation ## General Definitions Camera Operations are handled by sending JSON strings via MQTT or ZMQ according to the [Topic Structure](#topic-structure). All operations have to be implemented by the connector application as long as the operation is not marked as "optional". Proprietary operations and fields can be added according to vendors needs but must be prefixed by "cstm\_". Operations and fields defined in this standard must not start with this prefix. If a requested command or argument isn't supported by the vision connector (e.g., the command in the given Action field of the command is unknown) then the vision connector must reply with the return code 0x03000003 (see also [Error message codes](#error-message-codes)). ## Getting List of Supported Commands To get the full list of commands that are currently supported by the vision connector, the client can send following command. ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | ------ | ------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *GetSupportedCommands* | ```json { "TransactionID": "123456", "Action": "GetSupportedCommands" } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | -------------------- | -------- | ---------------- | ------------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | | SupportedCommandList | No | Array of Strings | Array of commands (the possible values for the Action field). | ```json { "TransactionID": "123456", "ReturnCode": 0, "Message": "Supported commands retrieved.", "SupportedCommandList": [ "EnumerateDevices", "GetStatus", "OpenDevice", "CloseDevice", "StartStreaming", "StopStreaming", "GetParameterNames", "GetParameters", "SetParameters", "SetQuickParameters", "SoftwareTrigger", "GetUserSets", "LoadUserSet", "SaveUserSet", "SetDefaultUserSet", "SetDeviceConfig", "GetDeviceConfig", "GetSupportedCommands" ] } ``` ## Camera Discovery As the initial step to establish communication with a camera, it is necessary to check the available cameras on the connector. To achieve this, the connector shall be capable of discovering cameras (e.g., GigE cameras) and listing them for the user. ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | ------ | --------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *EnumerateDevices* | ```json { "TransactionID": "123456", "Action": "EnumerateDevices" } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | ------------- | -------- | ----------------------- | ------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | | DeviceList | No | Array of Camera Devices | Array of camera devices. | ## Camera Object | Field | Optional | Type | Description | | ----------------------------------- | -------- | ------ | ------------------------------------------------------------------------------------------------- | | SerialNumber1 | Yes | String | Camera identification | | UserDefinedName1 | Yes | String | Camera User-defined name | | ModelName | No | String | Camera model | | VendorName | No | String | Camera vendor | | Interface | No | String | Camera interface *U3V* (USB3 Vision), *CXP* (CoaXPress), *GEV* (GigEVision), *CamEmu* (Emulation) | | IPAddress | Yes | String | Camera IP address | | MACAddress | Yes | String | Camera MAC address | | {additional identification fields}1 | Yes | String | Additional device identification offered by the connector application. | 1 Device identification, must be informed on the documentation if they can be used as device identification or if there is additional fields that can be used. ```json { "TransactionID": "123456", "ReturnCode": 0, "Message": "Cameras found.", "DeviceList": [ { "SerialNumber": "548451887", "UserDefinedName": "MyCam1", "ModelName": "Camera Model 1234", "VendorName": "The camera Factory", "Interface": "GEV", "IPAddress": "192.168.0.1", "MACAddress": "00:1A:2B:3C:4D:5E" } ] } ``` ## Open Camera Connection Usually, GenICam cameras demand the connection to be open before starting to set parameters and image streaming. This connection is an exclusive access, meaning that no other app can access the camera and change parameters or stream images. A camera can be opened by specifying an identification string in the DeviceID field. The connector will then try to find a matching camera by searching for this identification string amongst the infos of the available cameras. The order in which it searches is the following: 1. MACAddress 1. SerialNumber 1. IPAddress 1. UserDefinedName 1. ModelName 1. VendorName 1. Interface Example: An identification string is provided in the DeviceID field. Then the connector will first try to find a camera with a matching MACAddress. If none was found it will try to find a camera with a matching SerialNumber. It will continue with the other criteria until a matching camera can be found and try to open this camera. If multiple cameras are found matching the same info then the connector will try to open the first one. It is up to the vision connector which camera is the first one and may even be randomly chosen. If a camera was opened using a certain identification string then this identification string must be used in all DeviceID fields of all requests regarding this camera until it is closed again. A camera can't be opened with multiple identification strings at the same time. ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | ------ | --------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *OpenDevice* | | DeviceID | No | String | Device identification. | ```json { "TransactionID": "1232156", "Action": "OpenDevice", "DeviceID": "548451887" } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | ------------- | -------- | ------- | ------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | ```json { "TransactionID": "123456", "ReturnCode": 0, "Message": "Camera open." } ``` ## Close Camera Connection To close the camera connection, releasing the exclusive access and stopping any running image streaming, releasing the camera. It is *not* considered an error if no camera with the specified DeviceID is open or the camera has already been closed. However, an respective message is provided in the response. ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | ------ | ---------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *CloseDevice* | | DeviceID | No | String | Device identification. | ```json { "TransactionID": "1232116", "Action": "CloseDevice", "DeviceID": "548451887" } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | ------------- | -------- | ------- | ------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | ```json { "TransactionID": "1232116", "ReturnCode": 0, "Message": "Camera closed." } ``` ## Camera Parameter Configuration ## Retrieving Parameter Names A list of all parameter names is provided when requested by the application user by sending the following message. If namespaces are provided in the request then the parameter names in the response must be prefixed with the according namespace (see example below). Otherwise, no prefix will be added in the response. ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | --------------- | ---------------------------------------------------------------------------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *GetParameterNames* | | DeviceID | No | String | Device identification. | | NamespaceList | Yes | Array of String | Namespaces of the parameter lists to query the parameter names from (defaults to camera parameter list if omitted). Example: | ```json { "TransactionID": "2345645", "Action": "GetParameterNames", "DeviceID": "548451887", "NamespaceList": [ "@CameraDevice", "@CameraInstance" ] } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | ----------------- | -------- | --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | | ParameterNameList | No | Array of String | String array containing the full parameters names (including the namespace prefixes for the node map if namespaces were provided in the request) | ```json { "TransactionID": "2345645", "ReturnCode": 0, "Message": "Parameter names retrieved successfully", "ParameterNameList": [ "@CameraDevice/Width", "@CameraDevice/Height", "@CameraDevice/PixelFormat", "@CameraInstance/BufferHandlingMode", ... ] } ``` ## Retrieving Parameter List A list of parameters is provided when requested by the application user. To request the parameter list, the user must send the following message. ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | --------------- | ------------------------------------------ | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *GetParameters* | | DeviceID | No | String | Device identification. | | ParameterList | No | Array of String | The list of parameters that shall be read. | ```json { "TransactionID": "2345645", "Action": "GetParameters", "DeviceID": "548451887", "ParameterList": [ "Width", "Height" ] } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | ------------- | -------- | ---------------------- | -------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | | ParameterList | No | Parameter Object Array | Parameter Object Array containing the parameters details | ## Parameter Object | Field | Optional | Type | Description | | -------------- | -------- | --------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Name | No | String | Unique parameter name. | | DisplayName | Yes | String | Displayable name of the parameter for user interface. | | Type | No | String | Parameter value type. Possible values: *String*, *Integer*, *Float*, *Enumeration*, *Boolean*, *Command* | | Value | Yes | String/Int/Float/Bool | Current defined value. On enumerations, it is the string defined. Is not available if Type is Command or Readable is True. | | IntValue | Yes | Integer | On enumerations, it is the integer defined. | | Readable | No | Boolean | Flag indicating if the parameter can be read. Be aware that the readable flag depends of the current camera state. | | Writable | No | Boolean | Flag indicating if the parameter can be written. Be aware that the writable flag depends of the current camera state. | | Minimum | Yes | Int/Float | Minimum allowed value for numeric type parameters | | Maximum | Yes | Int/Float | Maximum allowed value for numeric type parameters | | Increment | Yes | Int/Float | Allowed increment for the parameter. | | Representation | Yes | String | Information on how this parameter should be represented in a user interface. This is only available for integer and float parameters. Possible values: *Linear*, *Logarithmic*, *PureNumber*, *Boolean*, *HexNumber*, *IPV4Address*, *MACAddress* | | Alias | Yes | String | Name of an alias parameter. An alias parameter represents the same functionality but in a different form (e.g., a float parameter could have an integer parameter as an alias). | | EnumEntries | Yes | Enum Entries Array | Array containing the entries available for the parameter. This is used by parameters that use enumerations. | ## Enum Entries | Field | Optional | Type | Description | | ----------- | -------- | ------- | ---------------------------------------------------------------- | | DisplayName | Yes | String | This represents the name of the property to the user | | Value | No | String | Parameter value that needs to be used to set the parameter value | | IntValue | No | Integer | Integer value of the enum entry | | Description | Yes | String | Enum entry description. | ```json { "TransactionID": "2345645", "ReturnCode": 0, "Message": "Parameters read successfully", "ParameterList": [ { "Name": "Width", "DisplayName": "Width", "Type": "Integer", "Value": 1024, "Writable": true, "Minimum": 1, "Maximum": 4096, "Increment": 1, "Representation": "Linear", "Alias": "ImageWidth" }, { "Name": "PixelFormat", "DisplayName": "Pixel Format", "Type": "Enumeration", "Value": "Mono8", "IntValue": 17301505, "Writable": true, "EnumEntries": [ { "DisplayName": "Mono 8", "Value": "Mono8", "IntValue": 17301505, "Description": "This enumeration value sets the pixel format to Mono 8." }, { "DisplayName": "Mono 10", "Value": "Mono10", "IntValue": 17825795, "Description": "This enumeration value sets the pixel format to Mono 10." }, { "DisplayName": "Mono 12", "Value": "Mono12", "IntValue": 17825797, "Description": "This enumeration value sets the pixel format to Mono 12." }, { "DisplayName": "Mono 16", "Value": "Mono16", "IntValue": 17825799, "Description": "This enumeration value sets the pixel format to Mono 16." } ] } ] } ``` ## Setting Camera Parameters To set the parameters, the user can send the message described in this chapter. It is possible to update a range of parameters; however, any error that may occur shall return the camera to the original state without applying any modification. Enumerations parameter types might be also defined using the human-readable value. For example, setting the pixel format using the value as Mono8, despite the enumeration integer value. For String and Integer parameters, the connector expects an exact match for the set value. For float parameters, a difference of no more than 1% is allowed due to floating-point comparison. If the provided value falls outside the expected range, the setting call will return an error. When using Command parameters, providing values has to result in an error message. The list of set values will be returned in the response message. ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | ---------------------- | ----------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *SetParameters* | | DeviceID | No | String | Device identification. | | ParameterList | No | Parameter Object Array | Object to encapsulate the parameters to update. | ## Parameter Object | Field | Optional | Type | Description | | ------ | -------------------------------------- | --------------------- | ------------------------------------------------------------------------------------------------------------------ | | Name | No | String | Unique Parameter Name2 | | Value1 | No / Yes (for command parameters only) | String/Int/Float/Bool | New value (the Value field must not be used for command parameters and must be used for all other parameter types) | 1On enumerations, it can be the integer value or can also be the symbolic name, refer to IntValue [Retrieving Parameter List](#retrieving-parameter-list). 2While it is recommended to use the unique parameter name, retrieved in [Retrieving Parameter List](#retrieving-parameter-list), applications may implement abbreviated parameter names as shortcuts to simplify the parameter settings or shortcuts to access multiple node maps. These shortcuts should be documented in the connector documentation. ```json { "TransactionID": "123567", "Action": "SetParameters", "DeviceID": "548451887", "ParameterList":[ {"Name": "Width", "Value": 800}, {"Name": "Height", "Value": 600}, {"Name": "PixelFormat", "Value": "Mono8"} //Setting using the human-readable string. This could be also the correspondent enum integer. ] } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | ------------- | -------- | ---------------------- | ------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | | ParameterList | No | Parameter Object Array | Object to encapsulate the parameters to update. | ## Parameter Object | Field | Optional | Type | Description | | ----- | -------- | --------------------- | ---------------------- | | Name | No | String | Unique Parameter Name2 | | Value | No | String/Int/Float/Bool | New value | ```json { "TransactionID": "123567", "ReturnCode": 0, "Message": "Parameters updated.", "ParameterList":[ {"Name": "Width", "Value": 800}, {"Name": "Height", "Value": 600}, {"Name": "PixelFormat", "Value": "Mono8"} ] } ``` ## Camera Quick Settings Additionally to the parameters settings, the quick setting feature provides a way to quickly define the most commonly used parameters. These parameters use common known names, and the connector is expected to map them internally to the camera correspondent parameter. ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | ---------------- | --------------------------------------------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *SetQuickParameters* | | DeviceID | No | String | Device identification. | | Width | Yes | Integer/String | Image width1 | | PixelFormat | Yes | Integer/String | Pixel format1,2 | | Height | Yes | Integer/String | Image Height1 | | Gain | Yes | Float/Int/String | Gain1. The string *Auto* can be used to activate automatic gain. | | ExposureTime | Yes | Float/Int/String | Exposure time in microseconds1. The string *Auto* can be used to activate automatic exposure. | 1 Camera dependent values, invalid values shall be informed on the returning error message. 2 GenICam standardized pixel format, according to the **GenICam Naming Convention** as described in section 4.35 [GenICam_PFNC_2_4.pdf](https://www.emva.org/wp-content/uploads/GenICam_PFNC_2_4.pdf). ```json { "TransactionID": "23456435", "Action": "SetQuickParameters", "DeviceID": "548451887", "Width": 800, "Height": 600, "Gain": "Auto", "PixelFormat": "Mono8", "ExposureTime": 234.5 } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | ------------- | -------- | ------- | ------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | ```json { "TransactionID": "23456435", "ReturnCode": 0, "Message": "Parameters updated." } ``` ## Camera User Settings The GenICam protocol allows the user to store parameter sets on the camera, and the connector must allow the user to save, load, and define the default user configuration to be used on camera restart. ## Get Available Saved User Sets Retrieve the available user sets on the camera. ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | ------ | ---------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *GetUserSets* | | DeviceID | No | String | Device identification. | ```json { "TransactionID": "1232116", "Action": "GetUserSets", "DeviceID": "12323454754" } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | ------------- | -------- | --------------------- | ------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | | UserSetList | No | User Set Object Array | Object containing the available parameter settings | ## User Set Object | Field | Optional | Type | Description | | ----------- | -------- | ------- | --------------------------------------------------------------- | | DisplayName | Yes | String | This represents the name of the user set to the user | | Value | No | String | Parameter value that needs to be used to set the user set value | | IntValue | No | Integer | Integer value of the user set | | Description | Yes | String | User set description. | ```json { "TransactionID": "12323454754", "ReturnCode": 0, "Message": "User set retrieved.", "UserSetList": [ { "DisplayName": "Default User Set", "Value": "Default", "IntValue": 0, "Description": "The default factory set can be loaded." }, { "DisplayName": "User Set 1", "Value": "UserSet1", "IntValue": 1, "Description": "User set 1 can be saved, loaded, or configured." }, { "DisplayName": "User Set 2", "Value": "UserSet2", "IntValue": 2, "Description": "User set 2 can be saved, loaded, or configured." }, { "DisplayName": "User Set 3", "Value": "UserSet3", "IntValue": 3, "Description": "User set 3 can be saved, loaded, or configured." } ] } ``` ## Load UserSet Configuration To load a configured UserSet on the camera, the user can send the *LoadUserSet* message. ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | ---------- | ------------------------------------------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *LoadUserSet* | | DeviceID | No | String | Device identification. | | UserSet | No | Int/String | String according with the user setting enumeration or integer when using the integer value. | ```json { "TransactionID": "23423423563456", "Action": "LoadUserSet", "DeviceID": "548451887", "UserSet": "UserSet2" } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | ------------- | -------- | ------- | ------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | ```json { "TransactionID": "1232116", "ReturnCode": 0, "Message": "User set loaded." } ``` ## Saving Configuration After the camera is parametrized, the parameter set can be saved on the camera on user sets. To perform the storage, the user can send the *SaveUserSet* message. ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | ---------- | ------------------------------------------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *SaveUserSet* | | DeviceID | No | String | Device identification. | | UserSet | No | Int/String | String according with the user setting enumeration or integer when using the integer value. | ```json { "TransactionID": "489494", "Action": "SaveUserSet", "DeviceID": "548451842342387", "UserSet": "UserSet2" } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | ------------- | -------- | ------- | ------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | ```json { "TransactionID": "489494", "ReturnCode": 0, "Message": "User set updated." } ``` ## Defining Default Configuration To define the camera default parameter set on the camera to be loaded in case of restart, the user can send the *SetDefaultUserSet* message. ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | ---------- | ------------------------------------------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *SetDefaultUserSet* | | DeviceID | No | String | Device identification. | | UserSet | No | Int/String | String according with the user setting enumeration or integer when using the integer value. | ```json { "TransactionID": "1232116", "Action": "SetDefaultUserSet", "DeviceID": "548451887", "UserSet": "UserSet2" } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | ------------- | -------- | ------- | ------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | ```json { "TransactionID": "1232116", "ReturnCode": 0, "Message": "Default user set defined." } ``` ## Download Camera Configuration The current camera setup might be downloaded to back up and further usages. The file format might be defined in the implementation, it is not meant to be edited externally. ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | ------ | -------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *GetDeviceConfig* | | DeviceID | No | String | Device identification. | ```json { "TransactionID": "23423456", "Action": "GetDeviceConfig", "DeviceID": "548451887" } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | ------------- | -------- | ------- | ------------------------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | | FileContent | No | String | Base64 string of the configuration file. Containing the mime type prefix. | ```json { "TransactionID": "23423456", "ReturnCode": 0, "Message": "Success", "FileContent": "data:text/plain;base64,IyB7MDVEOEMyOTQtRjI5NS00ZGZiLTlEMDEtMDk2QkQwNDA0OUY0fQojIEdlbkFwaSBwZXJzaXN0ZW5jZSBmaWxlICh2ZXJzaW9uIDMuMS4wKQojIERldmljZSA9IEJhc2xlcjo6QmFzbGVyQ2FtRW11IC0tIEJhc2xlciBDYW0gRW11IGludGVyZmFjZSAtLSBEZXZpY2UgdmVyc2lvbiA9IDEuMC4xIC0tIFByb2R1Y3QgR1VJRCA9IEI5N0RFNTQ5LUNCOUEtNDRmOC05MzdELTY0QzQ4QkRFQ0ZBMyAtLSBQcm9kdWN0IHZlcnNpb24gR1VJRCA9IEEyRENERTE0LTczRTQtNDA4YS1BRkE4LTgxODUyRUNCNUQ5QQpHYWluCTAuMDAwZSswMApXaWR0aAkxMDI0CkhlaWdodAkxMDQwClBpeGVsRm9ybWF0CU1vbm84CkV4cG9zdXJlVGltZQkxMDAwMC4wCg==" } ``` ## Upload Camera Configuration The stored file in section [Download Camera Configuration](#download-camera-configuration) can be provided to the camera restoring the setup. File integrity and validations are recommended and shall be implemented by the connector application. ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | ------ | ------------------------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *SetDeviceConfig* | | DeviceID | No | String | Device identification. | | FileContent | No | String | Base64 string of the configuration file. Containing the mime type prefix. | ```json { "TransactionID": "234423465465", "Action": "SetDeviceConfig", "DeviceID": "548451887", "FileContent": "data:text/plain;base64,IyB7MDVEOEMyOTQtRjI5NS00ZGZiLTlEMDEtMDk2QkQwNDA0OUY0fQojIEdlbkFwaSBwZXJzaXN0ZW5jZSBmaWxlICh2ZXJzaW9uIDMuMS4wKQojIERldmljZSA9IEJhc2xlcjo6QmFzbGVyQ2FtRW11IC0tIEJhc2xlciBDYW0gRW11IGludGVyZmFjZSAtLSBEZXZpY2UgdmVyc2lvbiA9IDEuMC4xIC0tIFByb2R1Y3QgR1VJRCA9IEI5N0RFNTQ5LUNCOUEtNDRmOC05MzdELTY0QzQ4QkRFQ0ZBMyAtLSBQcm9kdWN0IHZlcnNpb24gR1VJRCA9IEEyRENERTE0LTczRTQtNDA4YS1BRkE4LTgxODUyRUNCNUQ5QQpHYWluCTAuMDAwZSswMApXaWR0aAkxMDI0CkhlaWdodAkxMDQwClBpeGVsRm9ybWF0CU1vbm84CkV4cG9zdXJlVGltZQkxMDAwMC4wCg==" } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | ------------- | -------- | ------- | ------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | ```json { "TransactionID": "234423465465", "ReturnCode": 0, "Message": "Configuration loaded." } ``` ## Application Control The connector application shall allow the user to start and stop the image streaming. The control should not affect any camera property, including the trigger definitions. All camera parameter setups must be done through the parameter setup section. ## Starting Streaming This command is used to initiate image streaming for a specific camera. Upon activation, continuous streaming will publish images to the streaming topic (refer to [Topic Structure](#topic-structure)). For triggered grabbing, the application will send images based on trigger activations configured in the parameters section. If the optional FrameCount parameter is specified, the camera will stream the specified number of frames. It will automatically stop streaming when the frame count is reached (in this case, an event must also be sent to indicate that streaming has stopped, as described here [Stop Streaming](#stop-streaming)). If no value is specified for the FrameCount parameter, the camera will continue to stream until the user stops streaming. If a value less than 1 is specified for the FrameCount parameter, an error message must be sent. ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | ------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *StartStreaming* | | DeviceID | No | String | Device identification. | | FrameCount | Yes | Integer | Number of frames to be streamed. If no frame count is provided, the camera will stream continuously until the user stops the streaming. Values smaller than 1 are not valid and result in an error message | ```json { "TransactionID": "12341263412635465", "Action": "StartStreaming", "DeviceID": "548451887", "FrameCount": 100 } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | ------------- | -------- | ------- | ------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | ```json { "TransactionID": "12341263412635465", "ReturnCode": 0, "Message": "Stream started." } ``` ## Stop Streaming This command stops streaming images from the camera. It is *not* considered an error if the camera with the specified DeviceID is not currently streaming, or if streaming has already been stopped. However, an appropriate message should be returned in the response. When streaming is stopped (either automatically or via [Stop Streaming](#stop-streaming) command) the vision connector must notify this as an event using the message code 0x00000003 (see also [Info message codes](#info-message-codes) and [Device Specific Events](#device-specific-events)). ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | ------ | ------------------------------------------ | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *StopStreaming* | | DeviceID | No | String | Device identification. | ```json { "TransactionID": "32143242356465", "Action": "StopStreaming", "DeviceID": "548451887" } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | ------------- | -------- | ------- | ------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | ```json { "TransactionID": "32143242356465", "ReturnCode": 0, "Message": "Stream stopped." } ``` ## Software Trigger Image Acquisition This command is used when the camera is in software trigger operation. Upon execution, it triggers image acquisition as configured on the camera. Images will be published on the image streaming topic (refer to [Topic Structure](#topic-structure)). ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | ------ | -------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *SoftwareTrigger* | | DeviceID | No | String | Device identification. | ```json { "TransactionID": "2342546345", "Action": "SoftwareTrigger", "DeviceID": "548451887" } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | ------------- | -------- | ------- | ------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | ```json { "TransactionID": "2342546345", "ReturnCode": 0, "Message": "Trigger sent." } ``` ## Get Application Status Retrieves the connected camera list with status, streaming topic, application version, and implemented specification version. For compatibility purposes, both the legacy streaming topic (refer to the previous version of this standard) and the new streaming topic (refer to[ZMQ](#zmq)) are supported. It is recommended to return the new streaming topic in the GetStatus response. ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | ------ | -------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *GetStatus* | ```json { "TransactionID": "46543214635244", "Action": "GetStatus" } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | -------------------- | -------- | ---------------------- | ------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | | ApplicationName | Yes | String | Connector application name | | ApplicationVersion | No | String | Connector application version | | SpecificationVersion | No | String | Specification version of the connector | | DeviceList | No | Array of Camera Object | Array of camera objects | ## Camera Object | Field | Optional | Type | Description | | ----------------------------------- | -------- | ------ | ------------------------------------------------------------------------------------------------- | | DeviceID | No | String | Current camera identification | | StreamingTopic | No | String | Topic where the camera streams images | | Status | No | String | Camera status. Possible values: *Connected*, *Streaming*, *Error* | | SerialNumber1 | Yes | String | Camera identification | | UserDefinedName1 | Yes | String | Camera User-defined name | | ModelName1 | No | String | Camera model | | VendorName1 | No | String | Camera vendor | | Interface1 | No | String | Camera interface *U3V* (USB3 Vision), *CXP* (CoaXPress), *GEV* (GigEVision), *CamEmu* (Emulation) | | IPAddress1 | Yes | String | Camera IP address | | MACAddress | Yes | String | Camera MAC address | | {additional identification fields}1 | Yes | String | Additional device identification offered by the connector application. | 1 Device identification, must be informed on the documentation if they can be used as device identification or if there are additional fields that can be used. ```json { "TransactionID": "46543214635244", "ReturnCode": 0, "Message": "Success.", "ApplicationName": "Demo Vision Connector", "ApplicationVersion": "1.0.1", "SpecificationVersion": "1.1.0", "DeviceList": [ { "DeviceID": "548451887", "StreamingTopic": "device/548451887/stream", "Status": "Streaming", "SerialNumber": "548451887", "UserDefinedName": "MyCam1", "ModelName": "Camera Model 1234", "VendorName": "The camera Factory", "Interface": "GEV", "IPAddress": "192.168.0.1", "MACAddress": "00:1A:2B:3C:4D:5E" } ] } ``` ## Application Feedback ## Message Codes The following message codes are expected for the responses. There are standard message codes that are standardized by this document, and custom message codes that can be defined by each Vision Connector individually. Message codes are 32-bit integers that are encoded in the following way: | Bit offset (lsb \<< x) | Width (bits) | Description | | ---------------------- | ------------ | ------------------------------------------------------------------------------------------------------------------------------------- | | 0 | 20 | Index of the message code (starting at 0 for each log level) | | 20 | 3 | Reserved bits (must be set to 0) | | 23 | 1 | Must be set to 1 for custom message codes or 0 otherwise | | 24 | 2 | Log level. The following log levels exist at the moment: 0 = Info messages 1 = Warning messages 2 = Debug messages 3 = Error messages | | 26 | 6 | Reserved bits (must be set to 0) | ## Info message codes Value range for standard info messages: 0x00000000 ... 0x000FFFFF Value range for custom info messages: 0x00800000 ... 0x008FFFFF | Standard Message Codes | Meaning | | ---------------------- | ---------------------------------- | | 0x00000000 | Request was completed successfully | | 0x00000001 | Application is online | | 0x00000002 | Application is offline | | 0x00000003 | Streaming of a device is stopped | ## Warning message codes Value range for standard warning messages: 0x01000000 ... 0x010FFFFF Value range for custom warning messages: 0x01800000 ... 0x018FFFFF | Standard Message Codes | Meaning | | ---------------------- | -------------------------------------------- | | 0x01000000 | A request was only partially successful | | 0x01000001 | One or multiple corrupt frames were received | | 0x01000002 | One or multiple dropped frames were detected | ## Debug message codes Value range for standard debug messages: 0x02000000 ... 0x020FFFFF Value range for custom debug messages: 0x02800000 ... 0x028FFFFF ## Error message codes Value range for standard error messages: 0x03000000 ... 0x030FFFFF Value range for custom error messages: 0x03800000 ... 0x038FFFFF | Standard Message Codes | Meaning | | ---------------------- | -------------------------------------------------------- | | 0x03000000 | Error while parsing a request | | 0x03000001 | Error while handling a request | | 0x03000002 | An opened device was removed | | 0x03000003 | A feature (e.g., a command or argument) is not supported | ## Application Logging The application log is expected to be published on the logging topic, any general application log which is not a direct response to any given command, like application unhandled errors, camera error messages, and camera connectivity issues. The purpose of this log is to provide a human-readable application feedback that is not intended for interpretation by any software. ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | -------------------------------- | | 1 | topic | Logging topic | Binary String (UTF-8): *logging* | | 2 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | --------- | -------- | ------- | ------------------------------------------------------------------------------- | | Timestamp | Yes | String | Application host date and time on ISO8601 format, when the message was created. | | Code | Yes | Integer | Message code following [Message Codes](#message-codes). | | Level | No | String | Log level - *Debug*, *Info*, *Warning*, *Error* | | Message | No | String | Response message | ```json { "Timestamp":"2014-01-09T13:35:34.000000000+0100", "Code": 50331650, "Level":"Error", "Message":"Lost connection with myCam1" } ``` ## Application Events Application events are expected to be published on the event topic, any general application events that are not direct responses to a specific command, such as unhandled application errors, camera error messages, and camera connectivity problems. Unlike the application log, the purpose of these events is to provide notifications that can be interpreted by software. Certain message types (identified by the message code) may contain additional data. However, this is only required for the message codes listed below. The messages in the events are Vision Connector specific and must not be interpreted. ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | -------------------------------- | | 1 | topic | Logging topic | Binary String (UTF-8): *logging* | | 2 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | -------------- | --------------- | ---------------------- | ----------------------------------------------------------------------------------------------------------------------------- | | Timestamp | Yes | String | Application host date and time on ISO8601 format, when the message was created. | | Code | No | Integer | Message code following [Message Codes](#message-codes). | | Level | No | String | Log level - *Debug*, *Info*, *Warning*, *Error* | | Message | No | String | Response message | | AdditionalData | Depends on Code | Additional Data Object | Additional message code dependent data. AdditionalData does not exist unless explicitly specified for a certain message code. | ```json { "Timestamp":"2014-01-09T13:35:34.000000000+0100", "Code": 2, "Level":"Info", "Message":"The vision connector is offline now." } ``` ## Device Specific Events Device specific events must contain additional data that contains the entry "DeviceID", which will provide the device ID that this event happened on. List of all device specific events: - 0x00000003 | Streaming of a device is stopped - 0x03000002 | An opened device was removed ## Additional Data Object | Field | Optional | Type | Description | | -------- | -------- | ------ | ----------------------------- | | DeviceID | No | String | Current camera identification | Example event for a device connection loss: ```json { "Timestamp":"2014-01-09T13:35:34.000000000+0100", "Code": 50331650, "Level":"Error", "Message":"Lost connection with myCam1", "AdditionalData": { "DeviceID": "myCam1" } } ``` ## Frame Warning Events Frame warning events must contain additional data that contains the entry "DeviceID", which will provide the device ID that this event happened on. Furthermore, a "FrameCount" must be provided if more than 1 frame is affected. List of all frame warning events: - 0x01000001 | One or multiple corrupt frames were received - 0x01000002 | One or multiple dropped frames were received ## Additional Data Object | Field | Optional | Type | Description | | ---------- | -------- | ------- | ------------------------------------------------------------------------------------------------------ | | DeviceID | No | String | Current camera identification | | FrameCount | Yes | Integer | The number of images affected (if no frame count is provided then this is equal to a frame count of 1) | Example event for dropped frames: ```json { "Timestamp":"2014-01-09T13:35:34.000000000+0100", "Code": 16777218, "Level":"Warning", "Message":"3 dropped frames have been detected on myCam1", "AdditionalData": { "DeviceID": "myCam1", "FrameCount": 3 } } ``` # Image Connector Applications Interface Specification ## *Version: 1.1.2* ## Contents - [Interface Definition of Camera Connectors](#interface-definition-of-camera-connectors) - [Version Handling and Compatibility](#version-handling-and-compatibility) - [General Definitions](#general-definitions) - [Topic Structure](#topic-structure) - [ZMQ](#zmq) - [MQTT](#mqtt) - [Device Identification](#device-identification) - [Messaging](#messaging) - [Vision Connector Image Format](#vision-connector-image-format) - [Camera Operation](#camera-operation) - [General Definitions](#general-definitions) - [Getting List of Supported Commands](#getting-list-of-supported-commands) - [Camera Discovery](#camera-discovery) - [Open Camera Connection](#open-camera-connection) - [Close Camera Connection](#close-camera-connection) - [Camera Parameter Configuration](#camera-parameter-configuration) - [Retrieving Parameter Names](#retrieving-parameter-names) - [Retrieving Parameter List](#retrieving-parameter-list) - [Setting Camera Parameters](#setting-camera-parameters) - [Camera Quick Settings](#camera-quick-settings) - [Camera User Settings](#camera-user-settings) - [Get Available Saved User Sets](#get-available-saved-user-sets) - [Load UserSet Configuration](#load-userset-configuration) - [Saving Configuration](#saving-configuration) - [Defining Default Configuration](#defining-default-configuration) - [Download Camera Configuration](#download-camera-configuration) - [Upload Camera Configuration](#upload-camera-configuration) - [Application Control](#application-control) - [Starting Streaming](#starting-streaming) - [Stop Streaming](#stop-streaming) - [Software Trigger Image Acquisition](#software-trigger-image-acquisition) - [Get Application Status](#get-application-status) - [Application Feedback](#application-feedback) - [Message Codes](#message-codes) - [Info message codes](#info-message-codes) - [Warning message codes](#warning-message-codes) - [Debug message codes](#debug-message-codes) - [Error message codes](#error-message-codes) - [Application Logging](#application-logging) - [Application Events](#application-events) - [Device Specific Events](#device-specific-events) - [Frame Warning Events](#frame-warning-events) ## Interface Definition of Camera Connectors This document outlines the northbound interface (indicating north in the automation pyramid) of a camera connector application and vision-driven applications. It covers the configuration of a connected camera and the retrieval/streaming of images from this camera for further processing. The southbound interface of the camera connector is defined by the connector provider, dependent on the technology used (GigEVision, GenICam, USBVision, etc.), and is therefore not part of this document. ## Version Handling and Compatibility This document is versioned in a \.\.\ schema. While patch and minor version increments must keep backwards compatibility with earlier versions of the standard, this may not be the case with an increment in the major version. The patch version must be incremented with every change in the document and is mainly used during incremental development of new features. Bigger changes in the document, like the addition of new commands or a large number of small additions, could be grouped to an increase in the minor version. Regarding backwards compatibility, the following rules apply: - Changes in commands are considered backwards compatible as long as the use of the old command still works and causes the same result. - Changes in replies are considered backwards compatible if the interpretation of the reply without the additional entries is still valid as before. - Adding MQTT and ZMQ topics is considered backwards compatible. - Adding new Message Codes is considered backwards compatible. Messages with such codes can be ignored in older versions. If a feature, command, or definition in this document will be removed in a future standard version, it must be marked as being deprecated first and must stay present and usable for a defined amount of time until the feature may be removed. This would then cause a breaking change and therefore an increase in the major version. Proprietary commands, replies, and fields can be added according to vendors' needs but must be prefixed by "cstm\_" and must be compliant with the above-mentioned compatibility rules. Standard commands, replies, and fields must not start with this prefix. **Recommendation:** Proprietary fields in proprietary commands do not need to add the prefix additionally. ## General Definitions To facilitate data transfer between the camera connector and vision-driven applications, the currently supported communication channels are [ZeroMQ (ZMQ)](https://zeromq.org/) and MQTT. For enhanced integration, such as with Siemens Industrial Edge, supplementary interfaces like REST API could also be considered. The connector application must possess two **bind** ZMQ ports and/or **MQTT** configurable connection parameters. These connections will be utilized for setting up the application, configuring the camera, defining image streaming options, and requesting images via software triggers. Images must adhere to the [Vision Connector Image Format](#vision-connector-image-format). All used ports and servers must be configurable in the application configuration file. The ZMQ ports (defaults: 5555 to REQ/REP and 5556 to PUB/SUB) must be documented in the application documentation. In the case of MQTT connections, the server, port, user, password and root topic should be configurable within the application. The connector should be designed to support one or multiple cameras. This support must be thoroughly detailed in the application documentation and should include a dedicated topic structure for each camera. The scope of this document is to define the interface protocols between vision-driven applications and camera connectivity applications. Communication specifics, camera protocols, camera setup, and application features are beyond the scope of this document. All definitions in this document, if not explicitly stated as optional, are mandatory for compliant implementations. Below is a schematic representation of the defined scope: ## Topic Structure ## ZMQ When using ZMQ connections, the expected structure outlined below: - To send commands, utilize single-frame messages sent to the REQ/REP port. - The image streaming topic on the PUB/SUB port: **device/{DeviceID}/stream** - For receiving human-readable application feedback, subscribe to topic on the PUB/SUB port (see also [Application Logging](#application-logging)): **app/logging** - For receiving events which should be handled by the subscriber, subscribe to topic on the PUB/SUB port (see also [Application Events](#application-events)): **app/events** ## MQTT For MQTT connections, the topic structure is as follows: - To send commands, publish messages to: **{MqttRoot}/app/request** - For receiving application command responses on: **{MqttRoot}/app/response** - Subscribe to the topic for receiving human-readable application feedback: **{mqttRoot}/app/logging** - For receiving events which should be handled by the subscriber: **{MqttRoot}/app/events** The message content should be formatted in JSON, employing the message field names as property names for each call, as described in the subsequent chapters of this document. Proprietary topics can be added according to vendors' needs but must be prefixed by "cstm\_". Standard topics must not start with this prefix. ## Recommendation Proprietary (sub-)topics that fit into already standardized topics should be added to these as additional proprietary subtopics. If a new proprietary topic is introduced and is prefixed with "cstm\_", proprietary subtopics of it may not add the prefix additionally. Old topic structure from V1.0.0 is deprecated and will be removed in the next version. ## Device Identification To identify a device, the connector application should specify, in its [Camera Discovery](#camera-discovery) documentation, the allowed fields for identification. Typically, expected fields include the serial number (**SerialNumber** field), user identification (**UserDefinedName** field), IP address (for GigE devices, in the **IPAddress** field), and MAC address (for GigE devices, in the **MACAddress** field). While these fields are commonly used, they are not mandatory, may not be available, and the connector application may introduce additional or alternative identification fields. **Note:** The MAC address is typically delimited by ":" (e.g., 20:87:56:70:95:04). In the subsequent documentation, the term "DeviceID" refers to the identifying string that was used for a camera device upon opening it. This exact string will then be used for identifying the opened camera device while it is used and until it is closed again. Only after that the camera device can be opened and used again with a different DeviceID. The value of the DeviceID can be one of the fields returned during the device info (see more details in chapter [Open Camera Connection](#open-camera-connection)). Using a different DeviceID for a camera device while it is open, even if it refers to the same device, may lead to errors. Exceptions to this rule are allowed if documented by the application. DeviceIDs must not start with the proprietary suffix "cstm\_" to avoid conflicts. The following identifiers can't be used as a DeviceID: - "request" - "response" - "logging" - "app" - "device" ## Messaging The message exchange will be based on JSON messages. The subsequent sections will outline the minimum content for the message payload, where the fields are properties for the payload JSON, and any additional fields might be utilized as specified in the connector application documentation, in accordance with the vendor's requirements. Therefore, subscribers must make sure that their JSON parser accepts additional fields on top of the described ones. For ZMQ, the JSON string payload will be encoded into a UTF-8 single-frame message. In MQTT messages, the JSON payload serves as the content of the message to be sent. ## Vision Connector Image Format To standardize image transfer between camera streaming applications and image-driven applications, the following format shall be used for transferring images. The ZMQ message consists of a multipart message with each frame/part described below. This message format is prepared to support multiple stream cameras. ## Message Envelope | Frame | Field | Description | Type | | ------- | -------- | --------------------------------- | ---------------------------------------------------------------------------------------------- | | 1 | topic | Device identification. | Binary String (UTF-8) to identify the device in the following syntax: device/{DeviceID}/stream | | 2 | metadata | Metadata | Binary Metadata JSON (UTF-8) | | 3 | image | Image 1 | Binary | | 4 ... n | image | Image n – For multi-stream camera | Binary | ## Metadata JSON | Field | Optional | Type | Default | Description | | ------------ | -------- | ------------------------- | ------- | ---------------------------------------------------------------------------------------------------------------------------- | | version | No | String | 1 | Metadata format version | | count | No | Integer | 1 | Number of images in the message | | timestamp | No | String | -- | Date and Time string UTF-8 ISO8601. Application host datetime. | | customfields | Yes | String | -- | Custom field to be defined by the app creator and handled by the consumers. Might be a simple string or JSON string or else. | | detail | No | Image Detail Object Array | -- | Image details | ## Image Detail Object | Field | Optional | Type | Default | Description | | ----------- | -------------------------- | ------- | ------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | id | No | String | -- | Unique image identifier. Suggested mnemonic format like {camid}_{camstream}_{datetime} | | seq | No | Integer | -- | Sequential number for the camera image. Starting at 1, incremented by 1 on each frame grabbed. | | height | No / Yes (encoded formats) | Integer | -- | Image Height | | width | No / Yes (encoded formats) | Integer | -- | Image Width | | formatns | Yes | String | Genicam | Image format namespace. For GenICam pixel format use **Genicam** and **Compressed** for compressed formats. 2 | | format | No | String | -- | Image format 1 2 3 | | linepadding | Yes | Integer | 0 | Number of bytes added at the end of the byte stream to reach a multiple of 4 bytes. Eg. an image with 61 pixels, mono8, will have a line padding of 3, filling the stream to reach 64 bytes. | | timestamp | Yes | String5 | -- | Clock ticks informed by the camera.4 | | metadata6 | Yes | String | -- | Image additional metadata field to be defined by the app creator and handled by the consumers. Might be a simple string or JSON string or else. | 1 For cameras using a GenICam standardized pixel format, it is recommended to use the GenICam Naming Convention as described in section 4.35 [GenICam_PFNC_2_4.pdf](https://www.emva.org/wp-content/uploads/GenICam_PFNC_2_4.pdf). 2 Additional values can be supported once described in the application documentation. 3 Compressed formats shall be mentioned in the application documentation, being recommended in the fields corresponding to the file extension (e.g., "png", "jpg", "bmp"). 4 Timestamps provided by the cameras are Ticks count. This can be referencing the Unix epoch or the number of Ticks since the camera is on or count of Ticks informed/in sync with the PTP server. 5 A string might be used since, in the PTP standard, the Tick count is expected to be an 80-bit number. However, the current GenICam protocol returns a 64-bit integer. A string will be more reliable for future implementations. 6 Cameras capable and with PTP enabled must inform the PTP synchronization status in the metadata. E.g., ```json … "metadata": "{'ptpstatus': 'locked'}" … ``` ## Metadata Example ```json { "version": "1", "count": 2, "timestamp": "2014-01-09T13:35:34.000000000+0100", "customfields": "{'xfield1':1,'xfield2':'abc'}", "detail":[ { "id":"mycamera_0_2014-01-09T13:35:34.000000000+0100", "seq": 3, "height": 60, "width": 80, "format": "png", "formatns": "Compressed", "timestamp": "123456", "metadata": "{'batchid':'batch1','stream':'lowdef'}" }, { "id":"mycamera_1_2014-01-09T13:35:34.000000000+0100", "seq": 3, "height": 852, "width": 1280, "formatns": "Genicam", "format": "BGR8", "timestamp": "123456", "linepadding": 0, "metadata": "{'batchid':'batch1','stream':'raw'}" } ] } ``` ## Camera Operation ## General Definitions Camera Operations are handled by sending JSON strings via MQTT or ZMQ according to the [Topic Structure](#topic-structure). All operations have to be implemented by the connector application as long as the operation is not marked as "optional". Proprietary operations and fields can be added according to vendors needs but must be prefixed by "cstm\_". Operations and fields defined in this standard must not start with this prefix. If a requested command or argument isn't supported by the vision connector (e.g., the command in the given Action field of the command is unknown) then the vision connector must reply with the return code 0x03000003 (see also [Error message codes](#error-message-codes)). ## Getting List of Supported Commands To get the full list of commands that are currently supported by the vision connector, the client can send following command. ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | ------ | ------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *GetSupportedCommands* | ```json { "TransactionID": "123456", "Action": "GetSupportedCommands" } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | -------------------- | -------- | ---------------- | ------------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | | SupportedCommandList | No | Array of Strings | Array of commands (the possible values for the Action field). | ```json { "TransactionID": "123456", "ReturnCode": 0, "Message": "Supported commands retrieved.", "SupportedCommandList": [ "EnumerateDevices", "GetStatus", "OpenDevice", "CloseDevice", "StartStreaming", "StopStreaming", "GetParameterNames", "GetParameters", "SetParameters", "SetQuickParameters", "SoftwareTrigger", "GetUserSets", "LoadUserSet", "SaveUserSet", "SetDefaultUserSet", "SetDeviceConfig", "GetDeviceConfig", "GetSupportedCommands" ] } ``` ## Camera Discovery As the initial step to establish communication with a camera, it is necessary to check the available cameras on the connector. To achieve this, the connector shall be capable of discovering cameras (e.g., GigE cameras) and listing them for the user. ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | ------ | --------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *EnumerateDevices* | ```json { "TransactionID": "123456", "Action": "EnumerateDevices" } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | ------------- | -------- | ----------------------- | ------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | | DeviceList | No | Array of Camera Devices | Array of camera devices. | ## Camera Object | Field | Optional | Type | Description | | ----------------------------------- | -------- | ------ | ------------------------------------------------------------------------------------------------- | | SerialNumber1 | Yes | String | Camera identification | | UserDefinedName1 | Yes | String | Camera User-defined name | | ModelName | No | String | Camera model | | VendorName | No | String | Camera vendor | | Interface | No | String | Camera interface *U3V* (USB3 Vision), *CXP* (CoaXPress), *GEV* (GigEVision), *CamEmu* (Emulation) | | IPAddress | Yes | String | Camera IP address | | MACAddress | Yes | String | Camera MAC address | | {additional identification fields}1 | Yes | String | Additional device identification offered by the connector application. | 1 Device identification, must be informed on the documentation if they can be used as device identification or if there is additional fields that can be used. ```json { "TransactionID": "123456", "ReturnCode": 0, "Message": "Cameras found.", "DeviceList": [ { "SerialNumber": "548451887", "UserDefinedName": "MyCam1", "ModelName": "Camera Model 1234", "VendorName": "The camera Factory", "Interface": "GEV", "IPAddress": "192.168.0.1", "MACAddress": "00:1A:2B:3C:4D:5E" } ] } ``` ## Open Camera Connection Usually, GenICam cameras demand the connection to be open before starting to set parameters and image streaming. This connection is an exclusive access, meaning that no other app can access the camera and change parameters or stream images. A camera can be opened by specifying an identification string in the DeviceID field. The connector will then try to find a matching camera by searching for this identification string amongst the infos of the available cameras. The order in which it searches is the following: 1. MACAddress 1. SerialNumber 1. IPAddress 1. UserDefinedName 1. ModelName 1. VendorName 1. Interface Example: An identification string is provided in the DeviceID field. Then the connector will first try to find a camera with a matching MACAddress. If none was found it will try to find a camera with a matching SerialNumber. It will continue with the other criteria until a matching camera can be found and try to open this camera. If multiple cameras are found matching the same info then the connector will try to open the first one. It is up to the vision connector which camera is the first one and may even be randomly chosen. If a camera was opened using a certain identification string then this identification string must be used in all DeviceID fields of all requests regarding this camera until it is closed again. A camera can't be opened with multiple identification strings at the same time. ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | ------ | --------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *OpenDevice* | | DeviceID | No | String | Device identification. | ```json { "TransactionID": "1232156", "Action": "OpenDevice", "DeviceID": "548451887" } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | ------------- | -------- | ------- | ------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | ```json { "TransactionID": "123456", "ReturnCode": 0, "Message": "Camera open." } ``` ## Close Camera Connection To close the camera connection, releasing the exclusive access and stopping any running image streaming, releasing the camera. It is *not* considered an error if no camera with the specified DeviceID is open or the camera has already been closed. However, a respective message should be provided in the response. ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | ------ | ---------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *CloseDevice* | | DeviceID | No | String | Device identification. | ```json { "TransactionID": "1232116", "Action": "CloseDevice", "DeviceID": "548451887" } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | ------------- | -------- | ------- | ------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | ```json { "TransactionID": "1232116", "ReturnCode": 0, "Message": "Camera closed." } ``` ## Camera Parameter Configuration ## Retrieving Parameter Names A list of all parameter names is provided when requested by the application user by sending the following message. If namespaces are provided in the request then the parameter names in the response must be prefixed with the according namespace (see example below). Otherwise, no prefix will be added in the response. ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | --------------- | ---------------------------------------------------------------------------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *GetParameterNames* | | DeviceID | No | String | Device identification. | | NamespaceList | Yes | Array of String | Namespaces of the parameter lists to query the parameter names from (defaults to camera parameter list if omitted). Example: | ```json { "TransactionID": "2345645", "Action": "GetParameterNames", "DeviceID": "548451887", "NamespaceList": [ "@CameraDevice", "@CameraInstance" ] } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | ----------------- | -------- | --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | | ParameterNameList | No | Array of String | String array containing the full parameters names (including the namespace prefixes for the node map if namespaces were provided in the request) | ```json { "TransactionID": "2345645", "ReturnCode": 0, "Message": "Parameter names retrieved successfully", "ParameterNameList": [ "@CameraDevice/Width", "@CameraDevice/Height", "@CameraDevice/PixelFormat", "@CameraInstance/BufferHandlingMode", ... ] } ``` ## Retrieving Parameter List A list of parameters is provided when requested by the application user. To request the parameter list, the user shall send the following message. ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | --------------- | ------------------------------------------ | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *GetParameters* | | DeviceID | No | String | Device identification. | | ParameterList | No | Array of String | The list of parameters that shall be read. | ```json { "TransactionID": "2345645", "Action": "GetParameters", "DeviceID": "548451887", "ParameterList": [ "Width", "Height" ] } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | ------------- | -------- | ---------------------- | -------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | | ParameterList | No | Parameter Object Array | Parameter Object Array containing the parameters details | ## Parameter Object | Field | Optional | Type | Description | | -------------- | -------- | --------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Name | No | String | Unique parameter name. | | DisplayName | Yes | String | Displayable name of the parameter for user interface. | | Type | No | String | Parameter value type. Possible values: *String*, *Integer*, *Float*, *Enumeration*, *Boolean*, *Command* | | Value | Yes | String/Int/Float/Bool | Current defined value. On enumerations, it is the string defined. Is not available if Type is Command or Readable is True. | | IntValue | Yes | Integer | On enumerations, it is the integer defined. | | Readable | No | Boolean | Flag indicating if the parameter can be read. Be aware that the readable flag depends of the current camera state. | | Writable | No | Boolean | Flag indicating if the parameter can be written. Be aware that the writable flag depends of the current camera state. | | Minimum | Yes | Int/Float | For *Integer* and *Float* parameters: The minimum allowed value of the parameter. For *String* parameters: The minimum number of allowed bytes1 in the string (provided as an *Integer*). Usually this value will be 0. For all other parameter types: No minimum will be provided. | | Maximum | Yes | Int/Float | For *Integer* and *Float* parameters: The maximum allowed value of the parameter. For *String* parameters: The maximum number of allowed bytes1 in the string (provided as an *Integer*). For all other parameter types: No maximum will be provided. | | Increment | Yes | Int/Float | Allowed increment for the parameter. | | Representation | Yes | String | Information on how this parameter should be represented in a user interface. This is only available for integer and float parameters. Possible values: *Linear*, *Logarithmic*, *PureNumber*, *Boolean*, *HexNumber*, *IPV4Address*, *MACAddress* | | Alias | Yes | String | Name of an alias parameter. An alias parameter represents the same functionality but in a different form (e.g., a float parameter could have an integer parameter as an alias). | | EnumEntries | Yes | Enum Entries Array | Array containing the entries available for the parameter. This is used by parameters that use enumerations. | 1 The number of bytes in the string is the number of UTF-8 encoded bytes, not the number of characters. This means that a string with 3 characters, where one character is a 4-byte UTF-8 character, will have a length of 6 bytes. ## Enum Entries | Field | Optional | Type | Description | | ----------- | -------- | ------- | ---------------------------------------------------------------- | | DisplayName | Yes | String | This represents the name of the property to the user | | Value | No | String | Parameter value that needs to be used to set the parameter value | | IntValue | No | Integer | Integer value of the enum entry | | Description | Yes | String | Enum entry description. | ```json { "TransactionID": "2345645", "ReturnCode": 0, "Message": "Parameters read successfully", "ParameterList": [ { "Name": "Width", "DisplayName": "Width", "Type": "Integer", "Value": 1024, "Readable": true, "Writable": true, "Minimum": 1, "Maximum": 4096, "Increment": 1, "Representation": "Linear", "Alias": "ImageWidth" }, { "Name": "PixelFormat", "DisplayName": "Pixel Format", "Type": "Enumeration", "Value": "Mono8", "IntValue": 17301505, "Readable": true, "Writable": true, "EnumEntries": [ { "DisplayName": "Mono 8", "Value": "Mono8", "IntValue": 17301505, "Description": "This enumeration value sets the pixel format to Mono 8." }, { "DisplayName": "Mono 10", "Value": "Mono10", "IntValue": 17825795, "Description": "This enumeration value sets the pixel format to Mono 10." }, { "DisplayName": "Mono 12", "Value": "Mono12", "IntValue": 17825797, "Description": "This enumeration value sets the pixel format to Mono 12." }, { "DisplayName": "Mono 16", "Value": "Mono16", "IntValue": 17825799, "Description": "This enumeration value sets the pixel format to Mono 16." } ] } ] } ``` ## Setting Camera Parameters To set the parameters, the user can send the message described in this chapter. It is possible to update a range of parameters; however, any error that may occur shall return the camera to the original state without applying any modification. Enumerations parameter types might be also defined using the human-readable value. For example, setting the pixel format using the value as Mono8, despite the enumeration integer value. For String and Integer parameters, the connector expects an exact match for the set value. For float parameters, a difference of no more than 1% is allowed due to floating-point comparison. If the provided value falls outside the expected range, the setting call will return an error. When using Command parameters, providing values has to result in an error message. The list of set values will be returned in the response message. ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | ---------------------- | ----------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *SetParameters* | | DeviceID | No | String | Device identification. | | ParameterList | No | Parameter Object Array | Object to encapsulate the parameters to update. | ## Parameter Object | Field | Optional | Type | Description | | ------ | -------------------------------------- | --------------------- | ------------------------------------------------------------------------------------------------------------------ | | Name | No | String | Unique Parameter Name2 | | Value1 | No / Yes (for command parameters only) | String/Int/Float/Bool | New value (the Value field must not be used for command parameters and must be used for all other parameter types) | 1On enumerations, it can be the integer value or can also be the symbolic name, refer to IntValue [Retrieving Parameter List](#retrieving-parameter-list). 2While it is recommended to use the unique parameter name, retrieved in [Retrieving Parameter List](#retrieving-parameter-list), applications may implement abbreviated parameter names as shortcuts to simplify the parameter settings or shortcuts to access multiple node maps. These shortcuts should be documented in the connector documentation. ```json { "TransactionID": "123567", "Action": "SetParameters", "DeviceID": "548451887", "ParameterList":[ {"Name": "Width", "Value": 800}, {"Name": "Height", "Value": 600}, {"Name": "PixelFormat", "Value": "Mono8"} //Setting using the human-readable string. This could be also the correspondent enum integer. ] } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | ------------- | -------- | ---------------------- | ------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | | ParameterList | No | Parameter Object Array | Object to encapsulate the parameters to update. | ## Parameter Object | Field | Optional | Type | Description | | ----- | -------- | --------------------- | ---------------------- | | Name | No | String | Unique Parameter Name2 | | Value | No | String/Int/Float/Bool | New value | ```json { "TransactionID": "123567", "ReturnCode": 0, "Message": "Parameters updated.", "ParameterList":[ {"Name": "Width", "Value": 800}, {"Name": "Height", "Value": 600}, {"Name": "PixelFormat", "Value": "Mono8"} ] } ``` ## Camera Quick Settings Additionally to the parameters settings, the quick setting feature provides a way to quickly define the most commonly used parameters. These parameters use common known names, and the connector is expected to map them internally to the camera correspondent parameter. ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | ---------------- | --------------------------------------------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *SetQuickParameters* | | DeviceID | No | String | Device identification. | | Width | Yes | Integer/String | Image width1 | | PixelFormat | Yes | Integer/String | Pixel format1,2 | | Height | Yes | Integer/String | Image Height1 | | Gain | Yes | Float/Int/String | Gain1. The string *Auto* can be used to activate automatic gain. | | ExposureTime | Yes | Float/Int/String | Exposure time in microseconds1. The string *Auto* can be used to activate automatic exposure. | 1 Camera dependent values, invalid values shall be informed on the returning error message. 2 GenICam standardized pixel format, according to the **GenICam Naming Convention** as described in section 4.35 [GenICam_PFNC_2_4.pdf](https://www.emva.org/wp-content/uploads/GenICam_PFNC_2_4.pdf). ```json { "TransactionID": "23456435", "Action": "SetQuickParameters", "DeviceID": "548451887", "Width": 800, "Height": 600, "Gain": "Auto", "PixelFormat": "Mono8", "ExposureTime": 234.5 } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | ------------- | -------- | ------- | ------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | ```json { "TransactionID": "23456435", "ReturnCode": 0, "Message": "Parameters updated." } ``` ## Camera User Settings The GenICam protocol allows the user to store parameter sets on the camera, and the connector must allow the user to save, load, and define the default user configuration to be used on camera restart. ## Get Available Saved User Sets Retrieve the available user sets on the camera. ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | ------ | ---------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *GetUserSets* | | DeviceID | No | String | Device identification. | ```json { "TransactionID": "1232116", "Action": "GetUserSets", "DeviceID": "12323454754" } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | ------------- | -------- | --------------------- | ------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | | UserSetList | No | User Set Object Array | Object containing the available parameter settings | ## User Set Object | Field | Optional | Type | Description | | ----------- | -------- | ------- | --------------------------------------------------------------- | | DisplayName | Yes | String | This represents the name of the user set to the user | | Value | No | String | Parameter value that needs to be used to set the user set value | | IntValue | No | Integer | Integer value of the user set | | Description | Yes | String | User set description. | ```json { "TransactionID": "12323454754", "ReturnCode": 0, "Message": "User set retrieved.", "UserSetList": [ { "DisplayName": "Default User Set", "Value": "Default", "IntValue": 0, "Description": "The default factory set can be loaded." }, { "DisplayName": "User Set 1", "Value": "UserSet1", "IntValue": 1, "Description": "User set 1 can be saved, loaded, or configured." }, { "DisplayName": "User Set 2", "Value": "UserSet2", "IntValue": 2, "Description": "User set 2 can be saved, loaded, or configured." }, { "DisplayName": "User Set 3", "Value": "UserSet3", "IntValue": 3, "Description": "User set 3 can be saved, loaded, or configured." } ] } ``` ## Load UserSet Configuration To load a configured UserSet on the camera, the user can send the *LoadUserSet* message. ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | ---------- | ------------------------------------------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *LoadUserSet* | | DeviceID | No | String | Device identification. | | UserSet | No | Int/String | String according with the user setting enumeration or integer when using the integer value. | ```json { "TransactionID": "23423423563456", "Action": "LoadUserSet", "DeviceID": "548451887", "UserSet": "UserSet2" } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | ------------- | -------- | ------- | ------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | ```json { "TransactionID": "1232116", "ReturnCode": 0, "Message": "User set loaded." } ``` ## Saving Configuration After the camera is parametrized, the parameter set can be saved on the camera on user sets. To perform the storage, the user can send the *SaveUserSet* message. ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | ---------- | ------------------------------------------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *SaveUserSet* | | DeviceID | No | String | Device identification. | | UserSet | No | Int/String | String according with the user setting enumeration or integer when using the integer value. | ```json { "TransactionID": "489494", "Action": "SaveUserSet", "DeviceID": "548451842342387", "UserSet": "UserSet2" } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | ------------- | -------- | ------- | ------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | ```json { "TransactionID": "489494", "ReturnCode": 0, "Message": "User set updated." } ``` ## Defining Default Configuration To define the camera default parameter set on the camera to be loaded in case of restart, the user can send the *SetDefaultUserSet* message. ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | ---------- | ------------------------------------------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *SetDefaultUserSet* | | DeviceID | No | String | Device identification. | | UserSet | No | Int/String | String according with the user setting enumeration or integer when using the integer value. | ```json { "TransactionID": "1232116", "Action": "SetDefaultUserSet", "DeviceID": "548451887", "UserSet": "UserSet2" } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | ------------- | -------- | ------- | ------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | ```json { "TransactionID": "1232116", "ReturnCode": 0, "Message": "Default user set defined." } ``` ## Download Camera Configuration The current camera setup might be downloaded to back up and further usages. The file format might be defined in the implementation, it is not meant to be edited externally. ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | ------ | -------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *GetDeviceConfig* | | DeviceID | No | String | Device identification. | ```json { "TransactionID": "23423456", "Action": "GetDeviceConfig", "DeviceID": "548451887" } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | ------------- | -------- | ------- | ------------------------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | | FileContent | No | String | Base64 string of the configuration file. Containing the mime type prefix. | ```json { "TransactionID": "23423456", "ReturnCode": 0, "Message": "Success", "FileContent": "data:text/plain;base64,IyB7MDVEOEMyOTQtRjI5NS00ZGZiLTlEMDEtMDk2QkQwNDA0OUY0fQojIEdlbkFwaSBwZXJzaXN0ZW5jZSBmaWxlICh2ZXJzaW9uIDMuMS4wKQojIERldmljZSA9IEJhc2xlcjo6QmFzbGVyQ2FtRW11IC0tIEJhc2xlciBDYW0gRW11IGludGVyZmFjZSAtLSBEZXZpY2UgdmVyc2lvbiA9IDEuMC4xIC0tIFByb2R1Y3QgR1VJRCA9IEI5N0RFNTQ5LUNCOUEtNDRmOC05MzdELTY0QzQ4QkRFQ0ZBMyAtLSBQcm9kdWN0IHZlcnNpb24gR1VJRCA9IEEyRENERTE0LTczRTQtNDA4YS1BRkE4LTgxODUyRUNCNUQ5QQpHYWluCTAuMDAwZSswMApXaWR0aAkxMDI0CkhlaWdodAkxMDQwClBpeGVsRm9ybWF0CU1vbm84CkV4cG9zdXJlVGltZQkxMDAwMC4wCg==" } ``` ## Upload Camera Configuration The stored file in section [Download Camera Configuration](#download-camera-configuration) can be provided to the camera restoring the setup. File integrity and validations are recommended and shall be implemented by the connector application. ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | ------ | ------------------------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *SetDeviceConfig* | | DeviceID | No | String | Device identification. | | FileContent | No | String | Base64 string of the configuration file. Containing the mime type prefix. | ```json { "TransactionID": "234423465465", "Action": "SetDeviceConfig", "DeviceID": "548451887", "FileContent": "data:text/plain;base64,IyB7MDVEOEMyOTQtRjI5NS00ZGZiLTlEMDEtMDk2QkQwNDA0OUY0fQojIEdlbkFwaSBwZXJzaXN0ZW5jZSBmaWxlICh2ZXJzaW9uIDMuMS4wKQojIERldmljZSA9IEJhc2xlcjo6QmFzbGVyQ2FtRW11IC0tIEJhc2xlciBDYW0gRW11IGludGVyZmFjZSAtLSBEZXZpY2UgdmVyc2lvbiA9IDEuMC4xIC0tIFByb2R1Y3QgR1VJRCA9IEI5N0RFNTQ5LUNCOUEtNDRmOC05MzdELTY0QzQ4QkRFQ0ZBMyAtLSBQcm9kdWN0IHZlcnNpb24gR1VJRCA9IEEyRENERTE0LTczRTQtNDA4YS1BRkE4LTgxODUyRUNCNUQ5QQpHYWluCTAuMDAwZSswMApXaWR0aAkxMDI0CkhlaWdodAkxMDQwClBpeGVsRm9ybWF0CU1vbm84CkV4cG9zdXJlVGltZQkxMDAwMC4wCg==" } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | ------------- | -------- | ------- | ------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | ```json { "TransactionID": "234423465465", "ReturnCode": 0, "Message": "Configuration loaded." } ``` ## Application Control The connector application shall allow the user to start and stop the image streaming. The control should not affect any camera property, including the trigger definitions. All camera parameter setups must be done through the parameter setup section. ## Starting Streaming This command is used to initiate image streaming for a specific camera. Upon activation, continuous streaming will publish images to the streaming topic (refer to [Topic Structure](#topic-structure)). For triggered grabbing, the application will send images based on trigger activations configured in the parameters section. If the optional parameter FrameCount is provided, the camera will stream the number of frames defined. It will stop streaming automatically when the frame count is reached (in this case also an event must be sent to notify that streaming is stopped as described here [Stop Streaming](#stop-streaming)). If no value for the parameter FrameCount is provided, the camera will stream continuously until the user stops the streaming. If a value smaller than 1 is provided for the parameter FrameCount, an error message has to be sent. ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | ------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *StartStreaming* | | DeviceID | No | String | Device identification. | | FrameCount | Yes | Integer | Number of frames to be streamed. If no frame count is provided, the camera will stream continuously until the user stops the streaming. Values smaller than 1 are not valid and result in an error message | ```json { "TransactionID": "12341263412635465", "Action": "StartStreaming", "DeviceID": "548451887", "FrameCount": 100 } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | ------------- | -------- | ------- | ------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | ```json { "TransactionID": "12341263412635465", "ReturnCode": 0, "Message": "Stream started." } ``` ## Stop Streaming This command terminates camera image streaming. It is *not* considered an error if the camera with the specified DeviceID is currently not streaming or streaming was already stopped. However, an appropriate message should be provided in the response. When streaming is stopped (either automatically or via [Stop Streaming](#stop-streaming) command) the vision connector must notify this as an event using the message code 0x00000003 (see also [Info message codes](#info-message-codes) and [Device Specific Events](#device-specific-events)). ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | ------ | ------------------------------------------ | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *StopStreaming* | | DeviceID | No | String | Device identification. | ```json { "TransactionID": "32143242356465", "Action": "StopStreaming", "DeviceID": "548451887" } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | ------------- | -------- | ------- | ------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | ```json { "TransactionID": "32143242356465", "ReturnCode": 0, "Message": "Stream stopped." } ``` ## Software Trigger Image Acquisition This command is used when the camera is in software trigger operation. Upon execution, it triggers image acquisition as configured on the camera. Images will be published on the image streaming topic (refer to [Topic Structure](#topic-structure)). ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | ------ | -------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *SoftwareTrigger* | | DeviceID | No | String | Device identification. | ```json { "TransactionID": "2342546345", "Action": "SoftwareTrigger", "DeviceID": "548451887" } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | ------------- | -------- | ------- | ------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | ```json { "TransactionID": "2342546345", "ReturnCode": 0, "Message": "Trigger sent." } ``` ## Get Application Status Retrieves the connected camera list with status, streaming topic, application version, and implemented specification version. Currently, for the sake of compatibility, there exists the legacy streaming topic (see the previous version of this standard) and the new streaming topic (see [ZMQ](#zmq)). It is recommended to return the new streaming topic in the GetStatus response. ## Message Envelope For ZMQ Message | Frame | Field | Description | Type | | ----- | ------- | -------------- | --------------------- | | 1 | payload | Method payload | Binary String (UTF-8) | ## Payload | Field | Optional | Type | Description | | ------------- | -------- | ------ | -------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | Action | No | String | Method called. Fixed value *GetStatus* | ```json { "TransactionID": "46543214635244", "Action": "GetStatus" } ``` ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | --------------------- | | 1 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | -------------------- | -------- | ---------------------- | ------------------------------------------------------- | | TransactionID | Yes | String | Transaction identification. | | ReturnCode | No | Integer | Message code following [Message Codes](#message-codes). | | Message | Yes | String | Response message | | ApplicationName | Yes | String | Connector application name | | ApplicationVersion | No | String | Connector application version | | SpecificationVersion | No | String | Specification version of the connector | | DeviceList | No | Array of Camera Object | Array of camera objects | ## Camera Object | Field | Optional | Type | Description | | ----------------------------------- | -------- | ------ | ------------------------------------------------------------------------------------------------- | | DeviceID | No | String | Current camera identification | | StreamingTopic | No | String | Topic where the camera streams images | | Status | No | String | Camera status. Possible values: *Connected*, *Streaming*, *Error* | | SerialNumber1 | Yes | String | Camera identification | | UserDefinedName1 | Yes | String | Camera User-defined name | | ModelName1 | No | String | Camera model | | VendorName1 | No | String | Camera vendor | | Interface1 | No | String | Camera interface *U3V* (USB3 Vision), *CXP* (CoaXPress), *GEV* (GigEVision), *CamEmu* (Emulation) | | IPAddress1 | Yes | String | Camera IP address | | MACAddress | Yes | String | Camera MAC address | | {additional identification fields}1 | Yes | String | Additional device identification offered by the connector application. | 1 Device identification, must be informed on the documentation if they can be used as device identification or if there are additional fields that can be used. ```json { "TransactionID": "46543214635244", "ReturnCode": 0, "Message": "Success.", "ApplicationName": "Demo Vision Connector", "ApplicationVersion": "1.0.1", "SpecificationVersion": "1.1.0", "DeviceList": [ { "DeviceID": "548451887", "StreamingTopic": "device/548451887/stream", "Status": "Streaming", "SerialNumber": "548451887", "UserDefinedName": "MyCam1", "ModelName": "Camera Model 1234", "VendorName": "The camera Factory", "Interface": "GEV", "IPAddress": "192.168.0.1", "MACAddress": "00:1A:2B:3C:4D:5E" } ] } ``` ## Application Feedback ## Message Codes For the responses, the following message codes are expected. There are standard message codes that are standardized by this document and custom message codes that can be defined by each vision connector individually. Message codes are 32-bit integers that are encoded in the following way: | Bit offset (lsb \<< x) | Width (bits) | Description | | ---------------------- | ------------ | ------------------------------------------------------------------------------------------------------------------------------------- | | 0 | 20 | Index of the message code (starting at 0 for each log level) | | 20 | 3 | Reserved bits (must be set to 0) | | 23 | 1 | Must be set to 1 for custom message codes or 0 otherwise | | 24 | 2 | Log level. The following log levels exist at the moment: 0 = Info messages 1 = Warning messages 2 = Debug messages 3 = Error messages | | 26 | 6 | Reserved bits (must be set to 0) | ## Info message codes Value range for standard info messages: 0x00000000 ... 0x000FFFFF Value range for custom info messages: 0x00800000 ... 0x008FFFFF | Standard Message Codes | Meaning | | ---------------------- | ---------------------------------- | | 0x00000000 | Request was completed successfully | | 0x00000001 | Application is online | | 0x00000002 | Application is offline | | 0x00000003 | Streaming of a device is stopped | ## Warning message codes Value range for standard warning messages: 0x01000000 ... 0x010FFFFF Value range for custom warning messages: 0x01800000 ... 0x018FFFFF | Standard Message Codes | Meaning | | ---------------------- | -------------------------------------------- | | 0x01000000 | A request was only partially successful | | 0x01000001 | One or multiple corrupt frames were received | | 0x01000002 | One or multiple dropped frames were detected | ## Debug message codes Value range for standard debug messages: 0x02000000 ... 0x020FFFFF Value range for custom debug messages: 0x02800000 ... 0x028FFFFF ## Error message codes Value range for standard error messages: 0x03000000 ... 0x030FFFFF Value range for custom error messages: 0x03800000 ... 0x038FFFFF | Standard Message Codes | Meaning | | ---------------------- | -------------------------------------------------------- | | 0x03000000 | Error while parsing a request | | 0x03000001 | Error while handling a request | | 0x03000002 | An opened device was removed | | 0x03000003 | A feature (e.g., a command or argument) is not supported | ## Application Logging The application log is expected to be published on the logging topic, any general application log which is not a direct response to any given command, like application unhandled errors, camera error messages, and camera connectivity issues. The purpose of this log is to provide a human-readable application feedback that is not intended for interpretation by any software. ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | -------------------------------- | | 1 | topic | Logging topic | Binary String (UTF-8): *logging* | | 2 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | --------- | -------- | ------- | ------------------------------------------------------------------------------- | | Timestamp | Yes | String | Application host date and time on ISO8601 format, when the message was created. | | Code | Yes | Integer | Message code following [Message Codes](#message-codes). | | Level | No | String | Log level - *Debug*, *Info*, *Warning*, *Error* | | Message | No | String | Response message | ```json { "Timestamp":"2014-01-09T13:35:34.000000000+0100", "Code": 50331650, "Level":"Error", "Message":"Lost connection with myCam1" } ``` ## Application Events The application events are expected to be published on the event topic, any general application events which are not direct responses to any given command, like application unhandled errors, camera error messages, and camera connectivity issues. In contrast to the application log, the purpose of these events is to provide notifications that can be interpreted by software. Certain message types (distinguished by the message code) may carry additional data. However, this is only required for the message codes listed below. The messages in the events are vision connector specific and must not be interpreted. ## Response | Frame | Field | Description | Type | | ----- | ------- | --------------- | -------------------------------- | | 1 | topic | Logging topic | Binary String (UTF-8): *logging* | | 2 | payload | Response object | Binary String (UTF-8) | ## Response Object | Field | Optional | Type | Description | | -------------- | --------------- | ---------------------- | ----------------------------------------------------------------------------------------------------------------------------- | | Timestamp | Yes | String | Application host date and time on ISO8601 format, when the message was created. | | Code | No | Integer | Message code following [Message Codes](#message-codes). | | Level | No | String | Log level - *Debug*, *Info*, *Warning*, *Error* | | Message | No | String | Response message | | AdditionalData | Depends on Code | Additional Data Object | Additional message code dependent data. AdditionalData does not exist unless explicitly specified for a certain message code. | ```json { "Timestamp":"2014-01-09T13:35:34.000000000+0100", "Code": 2, "Level":"Info", "Message":"The vision connector is offline now." } ``` ## Device Specific Events Device specific events must contain additional data that contains the entry "DeviceID", which will provide the device ID that this event happened on. List of all device specific events: - 0x00000003 | Streaming of a device is stopped - 0x03000002 | An opened device was removed ## Additional Data Object | Field | Optional | Type | Description | | -------- | -------- | ------ | ----------------------------- | | DeviceID | No | String | Current camera identification | Example event for a device connection loss: ```json { "Timestamp":"2014-01-09T13:35:34.000000000+0100", "Code": 50331650, "Level":"Error", "Message":"Lost connection with myCam1", "AdditionalData": { "DeviceID": "myCam1" } } ``` ## Frame Warning Events Frame warning events must contain additional data that contains the entry "DeviceID", which will provide the device ID that this event happened on. Furthermore, a "FrameCount" must be provided if more than 1 frame is affected. List of all frame warning events: - 0x01000001 | One or multiple corrupt frames were received - 0x01000002 | One or multiple dropped frames were received ## Additional Data Object | Field | Optional | Type | Description | | ---------- | -------- | ------- | ------------------------------------------------------------------------------------------------------ | | DeviceID | No | String | Current camera identification | | FrameCount | Yes | Integer | The number of images affected (if no frame count is provided then this is equal to a frame count of 1) | Example event for dropped frames: ```json { "Timestamp":"2014-01-09T13:35:34.000000000+0100", "Code": 16777218, "Level":"Warning", "Message":"3 dropped frames have been detected on myCam1", "AdditionalData": { "DeviceID": "myCam1", "FrameCount": 3 } } ``` # Industrial Information Hub documentation # Contact ## Industrial Edge Community Your forum for information, knowledge exchange and solutions around Industrial Edge. Ask questions, post ideas, and stay up-to-date on Industrial Edge news. [Join the conversation](https://forum.mendix.com/link/space/industrial-edge) ## Experts Let our experts advise you! If you want to know more about Industrial Edge and potential applications in your company, just [talk to our experts](https://www.siemens.com/global/en/products/automation/topic-areas/industrial-edge.html#Letourexpertsadviseyou) – they’ll be happy to help. - [Overview](overview.html) - [IIH Essentials APIs](api/iih-essentials/overview.html) - [V2.5.0](api/iih-essentials/v2.5/iih-essentials-2.5.0.html) - [V2.4.0](api/iih-essentials/v2.4/iih-essentials-2.4.0.html) - [V2.3.0](api/iih-essentials/v2.3/iih-essentials-2.3.0.html) - [V2.2.0](api/iih-essentials/v2.2/iih-essentials-2.2.0.html) - [V2.1.0](api/iih-essentials/v2.1/iih-essentials-2.1.0.html) - [V2.0.0](api/iih-essentials/v2.0/iih-essentials-2.0.0.html) - [IIH Semantics APIs](api/iih-semantics/overview.html) - [V2.5.0](api/iih-semantics/v2.5/iih-semantics-2.5.0.html) - [V2.4.0](api/iih-semantics/v2.4/iih-semantics-2.4.0.html) - [V2.3.0](api/iih-semantics/v2.3/iih-semantics-2.3.0.html) - [Contact](contact.html) # Industrial Information Hub ## What is IIH? Industrial Information Hub (IIH) is an industrial data operations solution. It serves as an integration layer for industrial data and bridges the gap from the shop floor, through other levels, up to the cloud. Using the collected data from shop floor devices it provides information to other services and systems with its capabilities in structuring, transforming, storing and synchronizing the data. This enables access to data on all levels in a multitude of possibilities. The IIH can be used, together with other components, to integrate manufacturing data from the shopfloor all the way up to the cloud. Dive deeper into the details with our full [IIH user documentation](https://docs.industrial-operations-x.siemens.cloud/access?ft:title=Industrial%20Information%20Hub). Below, you'll find a quick summary of the components that make up IIH. ### Common Configurator Common Configurator represents the central user interface for Industrial Information Hub and all Industrial Edge connectors. With Common Configurator you can configure connectors, Industrial Edge Databus, IIH Essentials and IIH Semantics. Common Configurator can also be used outside the IIH with limited functionality. ### IIH Essentials IIH Essentials acts as the database for IIH. It provides configuration and storing of advanced models. Data received from connectors is structured using the model and stored in its IoT-Database. The data may be transformed and aggregated before providing it to clients through various interfaces. Its API is the basis for other apps like Performance Insight and many more. Additionally, IIH Essentials can actively synchronize the model and data to various clouds. Essentials can be used as a standalone-app for a deployment on small devices. ### IIH Semantics IIH Semantics adds semantic functionalities to the IIH solution. It enables querying the advanced model and data through GraphQL by using a knowledge graph. An OPC UA server makes it possible to share data and information within IIH with OPC UA clients. Similarly it is possible to connect to an OPC UA server, or load a companion spec for easy configuration and integration into existing OPC UA configurations. ### Connectors IIH enables different connectors to be configured on a user interface. There are a variety of connectors that work with IIH. IIH connectors are not covered by this documentation. You can find information on the connectors [here](https://support.industry.siemens.com/cs/ww/en/view/109817512). ## Basics of IIH ### Data flow Data can be collected through the southbound interfaces: MQTT via IE Databus, Connectivity Suite, OPC UA and WinCC Runtime through OpenPipe. OPC UA and OpenPIpe do not require a connector in IIH as it is integrated. To set up your data connection: 1. Configure a connector, such as a SIMATIC S7+ Connector, and select which tags should be available within the system 1. Map tags of the connector directly to an attribute in the asset model or create a transformed attribute on basis of the tags 1. After mapping, the data of the tags is available to all IIH features such as: - OPC UA server - Northbound synchronization - Knowledge graph - Historical data, available to clients like Performance Insight - ... #### Connection to OPC UA server If you want to establish a connection to an OPC server, install Common Configurator and IIH Semantics. The first step is to load a server structure for OPC UA. You can create your own structure using any modelling software for OPC UA, such as SiOME or UA Expert, and then import it into IIH. ### Connectivity Suite and IIH-compatible connectors The latest generation of connectors follows the Siemens "Connectivity Suite" standard, providing a central configuration in Common Configurator. In addition to Connectivity Suite connectors, there are older generation connectors available that do not follow the "Connectivity Suite" standard but still can be configured using Common Configurator. All other connectors are not compatible with the connector configuration of Common Configurator, but their data can still be used in IIH. ### APIs The IIH offers the following interfaces for other components to use: - REST API (IIH Essentials) - the interface to access the entire configuration of IIH Essentials, the asset model, time series data and alarms. - OPC UA Server (IIH Semantics) - the IIH Semantics offers an integrated OPC UA Server for read and write access to the configured model. - Graph QL API (IIH Semantics) - an API for retrieving model data as well as live and historical time series data via the GraphQL query language. # IIH Essentials APIs | API | Version | | ----------------------- | ---------------------------------------- | | IIH Essentials REST API | [V2.5.0](v2.5/iih-essentials-2.5.0.html) | | IIH Essentials REST API | [V2.4.0](v2.4/iih-essentials-2.4.0.html) | | IIH Essentials REST API | [V2.3.0](v2.3/iih-essentials-2.3.0.html) | | IIH Essentials REST API | [V2.2.0](v2.2/iih-essentials-2.2.0.html) | | IIH Essentials REST API | [V2.1.0](v2.1/iih-essentials-2.1.0.html) | | IIH Essentials REST API | [V2.0.0](v2.0/iih-essentials-2.0.0.html) | # IIH Essentials API 2.0.0 [Download OpenAPI Specification](iih-essentials.public-2.0.0.openapi.json) # IIH Essentials API 2.1.0 [Download OpenAPI Specification](iih-essentials.public-2.1.0.openapi.json) # IIH Essentials API 2.2.0 [Download OpenAPI Specification](iih-essentials.public-2.2.0.openapi.json) # IIH Essentials API 2.3.0 [Download OpenAPI Specification](iih-essentials-2.3.0-openapi.json) # IIH Essentials API 2.4.0 [Download OpenAPI Specification](iih-essentials-2.4.0-openapi.json) # IIH Essentials API 2.5.0 [Download OpenAPI Specification](iih-essentials-2.5.0-openapi.json) # IIH Semantics APIs | API | Version | | --------------- | --------------------------------------- | | OPC UA REST API | [V2.5.0](v2.5/iih-semantics-2.5.0.html) | | OPC UA REST API | [V2.4.0](v2.4/iih-semantics-2.4.0.html) | | OPC UA REST API | [V2.3.0](v2.3/iih-semantics-2.3.0.html) | # IIH Semantics API 2.3.0 [Download OpenAPI Specification](iih-semantics-2.3.0.openapi.json) # IIH Semantics API 2.4.0 [Download OpenAPI Specification](iih-semantics-2.4.0.openapi.json) # IIH Semantics API 2.5.0 [Download OpenAPI Specification](iih-semantics-2.5.0.openapi.json) # Industrial IoT Open Source Tools and Libraries documentation # Contact For additional questions, please contact us in the MindSphere Community on [community.sw.siemens.com](https://community.sw.siemens.com/s/global-search/%40uri?t=1684646354559#sort=relevancy&f:@commontopicnav=%5BMINDSPHERE%5D) or on [github.com/mindsphere](https://github.com/mindsphere). # Open Source Projects on GitHub Industrial IoT Open Source Tools and Libraries are contributions from the Industrial IoT community. These are demo applications, libraries and SDKs as well as useful connectivity tools which are maintained and released as open source software. Below is the list of all [Siemens Insights Hub and Industrial IoT open source projects on GitHub](https://github.com/mindsphere/). - [Overview](overview.html) - MindConnect-NodeJs - [Overview](mindconnect-nodejs/index.html) - [Proxy Support](mindconnect-nodejs/proxy-support.html) - [Troubleshooting](mindconnect-nodejs/troubleshooting.html) - [License](mindconnect-nodejs/license.html) - Agent Development - [Overview](mindconnect-nodejs/agent-development/index.html) - [Retrying Operations](mindconnect-nodejs/agent-development/retrying-operations.html) - [MindConnect Agent Methods](mindconnect-nodejs/agent-development/mindconnect-agent-docs.html) - [Starter Projects](mindconnect-nodejs/agent-development/starter-projects.html) - [Agent State Storage](mindconnect-nodejs/agent-development/agent-state-storage.html) - TypeScript SDK - [Overview](mindconnect-nodejs/sdk/index.html) - [Authorization](mindconnect-nodejs/sdk/auth.html) - [Using HttpAction](mindconnect-nodejs/sdk/using-http-action.html) - [Contibuting to SDK](mindconnect-nodejs/sdk/contributing-to-sdk.html) - Command Line Interface - [Overview](mindconnect-nodejs/cli/index.html) - [Setting Up the CLI](mindconnect-nodejs/cli/setting-up-the-cli.html) - [Development Proxy](mindconnect-nodejs/cli/development-proxy.html) - [Commands](mindconnect-nodejs/cli/index.html) - ./mindconnect-nodejs/cli/markdown-help/\*\*.md - Authentication Helper - [Overview](mindsphere-auth-helper/index.html) - [License](mindsphere-auth-helper/license.html) - MindConnect Node-RED Node - [Overview](node-red-contrib-mindconnect/index.html) - [Getting Started](node-red-contrib-mindconnect/getting-started.html) - [License](node-red-contrib-mindconnect/license.html) - [All Open Source Projects](github.html) - [Contact](contact.html) # Industrial IoT Open Source Tools and Libraries Industrial IoT Open Source Tools and Libraries are contributions from the Industrial IoT community. These are demo applications, libraries and SDKs as well as useful connectivity tools which are maintained and released as open source software. The source code for all libraries is available on [Github](github.html) and the packages can be downloaded via common package management tools. Info **Siemens Xcelerator** will fully incorporate **MindSphere** – our leading **industrial IoT** solution – into a wide range of industry solutions and modular cloud services. As a result, the industrial IoT capabilities of MindSphere will be available to the full Siemens portfolio, software and hardware alike, while at the same time enabling our customer and partners across the Siemens Xcelerator ecosystem to build their industry specific applications. Take a look at [github.com/mindsphere](https://github.com/mindsphere) for all Insights Hub and Industrial IOT Open Source Projects. ## MindConnect-NodeJs This is a nodejs library which can be used as a starting point for custom agent implementation in node.js. The library comes with support for TimeSeries, Events and File Upload and it can create and update the DataSourceConfiguration and the Mappings in Industrial IoT. The most recent feature is support for automatic mappings to the asset instances. [Learn more...](mindconnect-nodejs/index.html) ## TypeScript SDK TypeScript/JavaScript Community SDK for Industrial IoT APIs. It implements support for both browser (e.g. angular, react...) and backend development in node.js and it supports all different Industrial IoT authentication types (Cookies, UserCredentials, AppCredentials, ServiceCredentials, Industrial IoT Agents). It is packaged as part of @mindconnect/mindconnect-nodejs library. [Learn more...](mindconnect-nodejs/sdk/index.html) ## Command Line Interface Industrial IoT GUIs are great but some tasks are still better done from the command line. The Industrial IoT CLI is a power tool for experienced Industrial IoT administrators as it gives you scriptable access to common tasks like agent onboarding and offboarding, asset management, historical time series upload, user management and it also offers valuable tooling for software developers. It is packaged as part of @mindconnect/mindconnect-nodejs library. [Learn more...](mindconnect-nodejs/cli/index.html) ## MindConnect Node-RED Node A Node-RED node which can be used to upload the time series data, files and events to Industrial IoT. The node runs on x86, Raspberry PI, SIMATIC IOT 2040 and is also available as a docker container on docker hub. There are also live flow examples with flows sending e.g. the OPC UA or MQTT Data to Industrial IoT at ## Authentication Helper This chrome extension can be used to simplify the development tasks for which you need to use the SESSION and XSRF-TOKEN to access Industrial IoT APIs. It provides an easy way to copy the Industrial IoT authentication cookies to the clipboard without having to go to chrome developer tools. [Learn more...](mindsphere-auth-helper/index.html) # MindConnect-NodeJS ## Introduction This is a nodejs library which can be used as a starting point for custom agent implementation in nodejs. The library also contains useful tooling like [command line support](cli/index.html), which enables the user to easily import (historical) timeseries data, files and events to Insights Hub. There is also a [Typescript/JavaScript SDK](sdk/index.html) and a [development proxy](cli/development-proxy.html) which can be used to kickstart your development. - [MindConnect-NodeJs at GitHub](https://github.com/mindsphere/mindconnect-nodejs) - [MindConnect-NodeJs at npm](https://www.npmjs.com/package/@mindconnect/mindconnect-nodejs) ## Prerequisites You will need [Node.js](https://nodejs.org) which can be acquired from [nodejs.org](https://nodejs.org/en/download/). The library can be installed everywhere where nodejs is running ( Windows, Linux, MacOS, Raspbian...) ## Getting and Installing the MindConnect-NodeJS Library There are several ways to install the library. The most common one is via npm registry: Installing the library via [npm](https://www.npmjs.com/package/@mindconnect/mindconnect-nodejs) ```shell # install the latest stable library from the npm registry npm install @mindconnect/mindconnect-nodejs # install the latest alpha version from the npm registry npm install @mindconnect/mindconnect-nodejs@alpha ``` As an alternative you can also clone the [GitHub](https://github.com/mindsphere/mindconnect-nodejs) repository and install the file from the local file. ```shell # clone the repository and run in the library directory npm install npm pack # in your project directory run npm install mindconnect-...tgz --save ``` The release files are also available for [download](https://github.com/mindsphere/mindconnect-nodejs/releases). ## Legal This project has been released under an [Open Source License](license.html). The release may include and/or use APIs to Siemens’ or third parties’ products or services. In no event shall the project’s Open Source license grant any rights in or to these APIs, products or services that would alter, expand, be inconsistent with, or supersede any terms of separate license agreements applicable to those APIs. “API” means application programming interfaces and their specifications and implementing code that allows other software to communicate with or call on Siemens’ or third parties’ products or services and may be made available through Siemens’ or third parties’ products, documentations or otherwise. # MindConnect-NodeJS - License MIT License Copyright © 2019 Siemens AG Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. # MindConnect-NodeJS Proxy Support Set the `http_proxy` or `HTTP_PROXY` environment variable if you need to connect via proxy to Insights Hub. The settings are valid for the MindConnect Agents, SDK, CLI, the development proxy and also for Node-RED agent. ## BASH (Linux, git bash...) ```shell export HTTP_PROXY=http://localhost:8888 ``` ## Windows CMD ```text set "HTTP_PROXY=http://localhost:8888" ``` ## Windows PowerShell ```powershell $Env:HTTP_PROXY="http://localhost:8888" ``` # Troubleshooting Tips Most troubleshooting tips apply for both the mindconnect-nodejs library and the Node-RED node. ## Connection can't be established An error like this indicates that there is some problem with network connectivity between your computer and Insights Hub. ```text Network error occured request to https://.piam.eu1.mindsphere.io/oauth/token failed, reason: connect ECONNREFUSED ``` This can be caused by several reasons (and in the very rare case, Insights Hub could be offline) but the most common one is that your http proxy is not properly configured. The proxy configuration is set via environment variables: ```bash # set http proxy environment variable export HTTP_PROXY=http://localhost:8888 ``` If you are using node-RED make sure that the environment variable is created **before the node-RED process is started**, especially if you are running it as a service. Insights Hub Online Status: If you are convinced that your proxy settings are correctly set, it also might be that Insights Hub is undergoing occasional offline maintenance. The current status of all Insights Hub services is available at: ## The library/node are sending the data but there is no data in Fleet Manager This is probably caused by some misconfiguration in your data source configuration and/or in the mappings. Please use Agent Diagnostic Tool on your Insights Hub dashboard to troubleshoot the issue. If you are using node-RED node, there is a link leading directly to the diagnostic tool in the node-red configuration. ## The client secret has expired you will have to onboard your agent again This indicates that the agent authentication is not working anymore and that you will have to onboard the agent again. ```text the client secret has expired, you will have to onboard the agent again (possible cause for this error is invalid date/time on the device) ``` Here are some probable causes: - The agent hasn't called Insights Hub APIs for a long time If your agent is calling Insights Hub APIs only occasionally the authentication secrets might expire. In order to prevent that the Node-RED has an automatic keep alive feature which renews the client secrets even if no data is sent. If you are writing a custom agent make sure that you are doing something like this: ```javascript setInterval(async () => { await retry(5, () => agent.RenewToken()); }, 3600000); ``` - Invalid date/time on the device The date and time on the device where the agent is running is not configured properly. This can happen on e.g. virtual machines which are put into sleep mode, edge devices and gateways which don't have access to NTP server etc. Please make sure that the time is set properly. ## Error occurred during keep alive (in Node-RED) ```text Error occured during keep alive ``` If this error occurs in node red, the probable cause is the implementation of keep alive functionality paired with regular deployments. Prior to the version 3.7.0. the node wasn't clearing the old interval_id timer after redeployment. This could cause a state where two or more concurrent timers were trying to renew the agent token and sometimes the agent would get tangled in the different renewals. - Mitigation for versions before 3.7.0 Restarting the node-RED after deployment of a new configuration should mitigate the issue. - Fix in version 3.7.0 and beyond The version 3.7.0 clears the timers for async logging and token renewal on close similar to the function node of node-red. See also: ## How to reset agent configuration If you have problems with your agent: 1. Stop the agent. 1. Move or delete the content of the .mc folder (the json files with configuration and authentication settings). 1. Offboard the agent. 1. Create new settings for the mindconnect library. 1. copy the new settings to the node. ## Resetting the agent settings from version 3.7.0 in Node-RED Since version 3.7.0. it is possible to delete the content of the .mc/agentconfig.json file and the agent settings directly from the node. Press on the "delete local configuration" button on the node, confirm the dialog and redeploy the node. If you are having problems, it is a good idea to restart the Node-RED runtime completely ## Node-RED node 3.9.0 is not loading in chromium on Raspberry Pi If the node is not loading in your chromium browser on your Raspberry Pi, please either upgrade chromium to a newer version (83.0.\*) or upgrade the node to the version 3.9.1 which should also work in older chromium versions. (See also the [corresponding issue](https://github.com/mindsphere/node-red-contrib-mindconnect/issues/103) on GitHub.) # MindConnect-NodeJS - Agent Development - Getting Started The easiest way to start is to use the provided command line interface to create a starter project: ```bash # for typescript nodejs project run npx @mindconnect/mindconnect-nodejs starter-ts # for javascript nodejs project run npx @mindconnect/mindconnect-nodejs starter-js # if you have CLI installed # for typescript project mdsp starter-ts # for javascript project mdsp starter-js ``` ## TypeScript vs JavaScript All examples are in [typescript](https://www.typescriptlang.org/). They can be converted to javascript by removing the types. ```javascript // example in typescript const i: number = 0; // example in javascript const i = 0; ``` ## async/await vs Promises The examples use the async/await syntax for the promises (available in typescript and in javascript from EcmaScript8) ```javascript try { await agent.OnBoard(); console.log("agent was successfully onboarded"); } catch (err) { console.log(err); } ``` But you can also write code like this using the classical promise syntax: ```javascript agent.OnBoard() .then (function() { console.log("agent was successfully onboarded"; )} ).error(function(err){ console.log(err) }); ``` ## How to create a NodeJs Insights Hub agent The following steps describe the easiest way to test the library. You can of course create the required dependencies also programmatically via API calls. ### Step 0: Create an asset type and aspect types MindSphere V3 IoT model requires that you create an asset type and aspect types to describe your assets. For the example we will create an asset type of type Engine with two aspect types: Environment and Vibration. (Note that my tenant is called castidev, you will have to use your own tenant name) More information about [Insights Hub Data Model](https://siemens.mindsphere.io/en/docs/tutorials/asset-manager). ### Step 1: Create an asset Create an asset (In example it is called **AcmeMotor**) of type Engine in AssetManager for your data. ### Step 2: Create an agent of type MindConnectLib in Insights Hub Create an agent in Asset Manager of type MindConnectLib (core.mclib) create initial JSON token and store it to file (e.g. agentconfig.json) ```json { "content": { "baseUrl": "https://southgate.eu1.mindsphere.io", "iat": "", "clientCredentialProfile": ["SHARED_SECRET"], "clientId": "a3ac5ae889544717b02fa8282a30d1b4", "tenant": "" }, "expiration": "2018-04-06T00:47:39.000Z" } ``` More Information about [MindConnectLib](http://bit.ly/2HZ2ehE) configuration. ### Step 3 : Create an agent Read the initial configuration from the config file and create the agent. If you are using the **SHARED_SECRET** profile there is no need to setup the local certificate for the communication (recommended for smaller devices). ```javascript const configuration = require("../../agentconfig.json"); const agent = new MindConnectAgent(configuration); ``` If you want to use the **RSA_3072** profile you must also set up the agent certificate. ```javascript // you can create the private.key for example using openssl: // openssl genrsa -out private.key 3072 agent.SetupAgentCertificate(fs.readFileSync("private.key")); ``` ### Step 4: Onboard the agent The first operation is onboarding of the agent. This creates a client secret which is used for the communication with Insights Hub. This data is stored by default in the .mc folder in your application if you don't change the base path in the constructor of the agent. **Important**: Make sure that your folder with the configurations is not reachable from the internet as it contains the client_secret for the authentication. ```javascript if (!agent.IsOnBoarded()) { await agent.OnBoard(); } ``` ### Step 5: Configure the data model and data mappings to asset variables. (via code) **Important**: From the version 3.8.0 it is possible to create the data source configuration and data mappings fully automatic via code First you need to create a data source configuration ```javascript // create data source configuration for an asset type castidev.Engine const generatedConfig = await agent.GenerateDataSourceConfiguration(`${agent.GetTenant()}.Engine`); await agent.PutDataSourceConfiguration(generatedConfig); ``` and then the data mappings: ```javascript const mappings = await agent.GenerateMappings(targetAssetId); await agent.PutDataMappings(mappings); ``` ### Step 5a: Create automatic mapping to target asset If you are just creating 1:1 asset to agent mappings the two steps above can be shortened into: ```javascript /// instead of creating the data source configuration and mappings separately // you can also just use the method below which takes care of everything // this is only used for 1:1 asset -> agent mappings // the method above can also map the data source configuration to multiple assets // just call GenerateMappings with different asset ids await agent.ConfigureAgentForAssetId(targetAssetId); ``` ### SDK access: The agents have now access to Industrial IoT TypeScript SDK ```javascript agent.Sdk(); // the sdk gives you access to e.g. asset management client with which you can get asset types or assets from Insights Hub // which can be used for automatic data source configuration and automatic mappings const assetMgmt = agent.Sdk().GetAssetManagementClient(); await assetMgmt.GetAssets(...); await assetMgmt.GetAspectTypes(...); ``` If you take a look at the Insights Hub configuration of your agent now it should look like this: And the data mappings should be in place ### Step 6 After this you can send the data in the code ```javascript for (let index = 0; index < 5; index++) { const values: DataPointValue[] = [ { dataPointId: "DP-Temperature", qualityCode: "0", value: (Math.sin(index) * (20 + (index % 2)) + 25).toString(), }, { dataPointId: "DP-Pressure", qualityCode: "0", value: (Math.cos(index) * (20 + (index % 25)) + 25).toString(), }, { dataPointId: "DP-Humidity", qualityCode: "0", value: ((index + 30) % 100).toString() }, { dataPointId: "DP-Acceleration", qualityCode: "0", value: (1000.0 + index).toString() }, { dataPointId: "DP-Frequency", qualityCode: "0", value: (60.0 + index * 0.1).toString() }, { dataPointId: "DP-Displacement", qualityCode: "0", value: (index % 10).toString() }, { dataPointId: "DP-Velocity", qualityCode: "0", value: (50.0 + index).toString() }, ]; // there is an optional timestamp parameter if you need to use something else instead of Date.now() const result = await agent.PostData(values); } ``` (If you were using UI to configure data mappings you will have long integers instead of human-readable data point Ids.) ### Step 6.1 using bulk upload If you don't want to send the data points one by one, you can also use the BulkPostData method ```javascript const bulk: TimeStampedDataPoint[] = [ { timestamp: "2018-08-23T18:38:02.135Z", values: [ { dataPointId: "DP-Temperature", qualityCode: "0", value: "10" }, { dataPointId: "DP-Pressure", qualityCode: "0", value: "10" }, ], }, { timestamp: "2018-08-23T19:38:02.135Z", values: [ { dataPointId: "DP-Temperature", qualityCode: "0", value: "10" }, { dataPointId: "DP-Pressure", qualityCode: "0", value: "10" }, ], }, ]; await agent.BulkPostData(bulk); ``` ## Events Events can now be created with the library. You can create events for your agent or for your entities. In order to create an event for your entity you need to know the assetId of the asset. ```javascript const configuration = require("../../agentconfig.json"); const agent = new MindConnectAgent(configuration); if (!agent.IsOnBoarded()) { await agent.OnBoard(); } const event: Insights HubStandardEvent = { entityId: configuration.content.clientId, // use assetid if you dont want to store event in the agent :) sourceType: "Event", sourceId: "application", source: "Meowz", severity: 20, // 0-99 : 20:error, 30:warning, 40: information timestamp: new Date().toISOString(), description: "Test", }; // send event with current timestamp await agent.PostEvent(event); ``` ## File Upload Files can now be uploaded via the library. You can upload files for your agent or for your entities. In order to create an event for your entity you need to know the assetId of the asset. Since version 3.5.1. the agents are using the multipart upload API of the Insights Hub. This means that the agents can upload files also bigger > 8 MB, The multipart upload must be switched on (chunk:true) if you want to activate this behavior. The parameter parallelUploads determine the maximal number of parallel uploads. You can increase this on a powerful computer to speed up the upload or decrease to prevent network congestion. ```javascript const configuration = require("../../agentconfig.json"); const agent = new MindConnectAgent(configuration); if (!agent.IsOnBoarded()) { await agent.OnBoard(); } await agent.UploadFile(agent.ClientId(), "custom/mindsphere/path/package.json", "package.json", { retry: RETRYTIMES, description: "File uploaded with MindConnect-NodeJS Library", parallelUploads: 5, chunk: true, }); ``` ## Full Agent Here is a demo agent implementation. [mindsphere-agent](https://github.com/mindsphere/mindconnect-nodejs/blob/master/src/demoagent/test-agent.ts) You can observe the data in the Insights Hub. # Mindconnect-NodeJS - Agent Development - Agent State Storage ## Introduction The agents need to manage their state information over time. This state information consists of - agent secrets - agent configuration - agent mappings - history of agent secrets - list of uploaded files and corresponding eTags (so that agents can overwrite the files if necessary) This information needs to be persisted over time. Info The agents store the eTag of the file so that they can overwrite these files in the future. This will change once agents can read the eTags of the files. ## Storage Provider - Default Implementation The default implementation stores the state in the hidden .mc directory in the root directory of your project. ```bash ls -la .mc total 8 drwxr-xr-x 1 sn0wcat 1049089 0 May 17 17:41 . drwxr-xr-x 1 sn0wcat 1049089 0 May 17 17:41 .. -rw-r--r-- 1 sn0wcat 1049089 2814 May 17 17:41 8ed19ef5515542b4bb05842bfbd48f38.json ``` ## Storing the data using different storage provider If you need to store the data in a different or more secure fashion you can provide your own implementation of the StorageProvider. You need to implement the following interface ```javascript /** * Per default, the library stores the agent settings in the directory .mc * You can pass a class which implements a ConfigurationStorage in the constructor if you want to store * the settings somewhere else. (e.g. database, encrypted file system etc) * @export * @interface ConfigurationStorage */ export interface IConfigurationStorage { GetConfig(config: IMindConnectConfiguration): IMindConnectConfiguration; SaveConfig(config: IMindConnectConfiguration): Promise; } ``` The GetConfig method should check if the config has changed and return no value so that the agent can be onboarded again ```javascript if (_.isEqual(json.content, configuration.content)) { return json; } else { log("The configuration has changed we will onboard again."); } ``` ### Example ```javascript export class MySecureStorage implements IConfigurationStorage { private lock: AsyncLock; private encrypt(): string { //... your secure implementation } private decrypt(): string { //... your secure implementation } public GetConfig(configuration: IMindConnectConfiguration): IMindConnectConfiguration { try { const json = ( const result = fs.readFileSync (path.resolve(`${this._basePath}/${configuration.content.clientId}.bin`)); return this.decrypt(result); ); if (_.isEqual(json.content, configuration.content)) { return json; } else { log("The configuration has changed we will onboard again."); } } catch (err) { log(`There is no configuration stored yet for agent with id ${configuration.content.clientId}`); } return configuration; } public async SaveConfig(config: IMindConnectConfiguration): Promise { const fileName = `${this._basePath}/${config.content.clientId}.bin`; return await this.lock.acquire(fileName, () => { const data = JSON.stringify(config); fs.writeFileSync(fileName, this.encrypt(data)); return config; }); } constructor(private _basePath: string) { if (!fs.existsSync(this._basePath)) { fs.mkdirSync(this._basePath); } this.lock = new AsyncLock({}); } } ``` ## Using your storage You can pass the instance of your storage provider to the MindConnectAgent constructor. ```javascript const agent = new MindConnectAgent(configuration, undefined, new MySecureStorage()); ``` # MindConnect-NodeJS - Agent Development - MindConnect Agent Methods MindConnect Agent implements the V3 of the [MindConnect API](https://documentation.mindsphere.io/MindSphere/apis/connectivity-mindconnect/api-mindconnect-overview.html#mindconnect-api). The synchronous methods (`IsOnBoarded`, `HasConfiguration`, `HasDataMapping`...) are operating on agent state storage only. The asynchronous methods (`GetDataSourceConfiguration`, `BulkPostData`...) are calling Insights Hub APIs. ## IsOnBoarded ```javascript /** * * Check in the local storage if the agent is onboarded. * * * This is a local agent state storage setting only. * * @returns {boolean} * @memberof MindConnectAgent */ public IsOnBoarded(): boolean ``` ## HasDataSourceConfiguration ```javascript /** * Checks in the local storage if the agent has a data source configuration. * * * This is a local agent state storage setting only. * * Call await GetDataSourceConfiguration() if you want to check if there is configuration in Insights Hub. * * * @returns {boolean} * @memberof MindConnectAgent */ public HasDataSourceConfiguration(): boolean ``` ## HasDataMappings ```javascript /** * Checks in the local storage if the agent has configured mappings. * * * This is a local agent state storage setting only. Insights Hub API is not called. * * Call await GetDataMappings() to check if the agent has configured mappings in the Insights Hub. * * @returns {boolean} * @memberof MindConnectAgent */ public HasDataMappings(): boolean ``` ## PutDataSourceConfiguration ```javascript /** * Stores the configuration in Insights Hub. * * By default the eTag parameter in the provided configuration is ignored, and the agent just updates the configuration every time the put method is stored * and automatically increases the eTag. * This is why its a good idea to check if the configuration was stored before the data was posted. If the ignoreEtag is set to false then the agent just uses * the eTag which was specified in the configuration. This might throw an "already stored" exception in Insights Hub. * * @param {DataSourceConfiguration} dataSourceConfiguration * @param {boolean} [ignoreEtag=true] * @returns {Promise} * @memberof MindConnectAgent */ public async PutDataSourceConfiguration( dataSourceConfiguration: DataSourceConfiguration, ignoreEtag: boolean = true ): Promise ``` ## GetDataSourceConfiguration ```javascript /** * Acquire DataSource Configuration and store it in the Agent Storage. * * @returns {Promise} * * @memberOf MindConnectAgent */ public async GetDataSourceConfiguration(): Promise ``` ## GetDataMappings ```javascript /** * Acquire the data mappings from Insights Hub and store them in the agent state storage. * * @returns {Promise>} * * @memberOf MindConnectAgent */ public async GetDataMappings(): Promise> ``` ## PutDataMappings ```javascript /** * Store data mappings in the Insights Hub and also in the local agent state storage. * * @param {Mapping[]} mappings * @returns {Promise} * * @memberOf MindConnectAgent */ public async PutDataMappings(mappings: Mapping[]): Promise ``` ## DeleteAllMappings ```javascript /** * Deletes all mappings from the agent * * @memberOf MindConnectAgent */ public async DeleteAllMappings(); ``` ## PostEvent ```javascript /** * Posts the Events to the Exchange Endpoint * * @param {*} events * @param {Date} [timeStamp=new Date()] * @param {boolean} [validateModel=true] * @returns {Promise} * @memberof MindConnectAgent */ public async PostEvent event: BaseEvent | CustomEvent, timeStamp: Date = new Date(), validateModel: boolean = true ): Promise ``` ## Post Data ```javascript /** * Post Data Point Values to the Exchange Endpoint * * @param {DataPointValue[]} dataPoints * @param {Date} [timeStamp=new Date()] * @param {boolean} [validateModel=true] you can set this to false to speed up the things if your agent is working. * @returns {Promise} * @memberof MindConnectAgent */ public async PostData( dataPoints: DataPointValue[], timeStamp: Date = new Date(), validateModel: boolean = true ): Promise ``` ## BulkPostData ```javascript /** * Post Bulk Data Point Values to the Exchange Endpoint. * * @param {TimeStampedDataPoint[]} timeStampedDataPoints * @param {boolean} [validateModel=true] * @returns {Promise} * * @memberOf MindConnectAgent */ public async BulkPostData( timeStampedDataPoints: TimeStampedDataPoint[], validateModel: boolean = true ): Promise ``` ## UploadFile ```javascript /** * Upload file to Insights Hub IOTFileService * * * This method is used to upload the files to the Insights Hub. * * It supports standard and multipart upload which can be configured with the [optional.chunk] parameter. * * * The method will try to abort the multipart upload if an exception occurs. * * Multipart Upload is done in following steps: * * start multipart upload * * upload in parallel [optional.parallelUploadChunks] the file parts (retrying [optional.retry] times if configured) * * uploading last chunk. * * @param {string} entityId - asset id or agent.ClientId() for agent * @param {string} filepath - Insights Hub file path * @param {(string | Buffer)} file - local path or Buffer * @param {fileUploadOptionalParameters} [optional] - optional parameters: enable chunking, define retries etc. * @param {(number | undefined)}[optional.part] multipart/upload part * @param {(Date | undefined)} [optional.timestamp] File timestamp in Insights Hub. * @param {(string | undefined)} [optional.description] Description in Insights Hub. * @param {(string | undefined)} [optional.type] Mime type in Insights Hub. * @param {(number | undefined)} [optional.chunkSize] chunkSize. It must be bigger than 5 MB. Default 8 MB. * @param {(number | undefined)} [optional.retry] Number of retries * @param {(Function | undefined)} [optional.logFunction] log functgion is called every time a retry happens. * @param {(Function | undefined)} [optional.verboseFunction] verboseLog function. * @param {(boolean | undefined)} [optional.chunk] Set to true to enable multipart uploads * @param {(number | undefined)} [optional.parallelUploads] max paralell uploads for parts (default: 3) * @param {(number | undefined)} [optional.ifMatch] The etag for the upload. * @returns {Promise} - md5 hash of the file * * @memberOf MindConnectAgent * * @example await agent.UploadFile (agent.GetClientId(), "some/mindsphere/path/file.txt", "file.txt"); * @example await agent.UploadFile (agent.GetClientId(), "some/other/path/10MB.bin", "bigFile.bin",{ chunked:true, retry:5 }); */ public async UploadFile( entityId: string, filepath: string, file: string | Buffer, optional?: fileUploadOptionalParameters ): Promise ``` ## GenerateDataSourceConfiguration ```javascript /** * Generates a Data Source Configuration for specified Asset Type * * you still have to generate the mappings (or use ConfigureAgentForAssetId method) * * @example * config = await agent.GenerateDataSourceConfiguration("castidev.Engine"); * * @param {string} assetTypeId * @param {("NUMERICAL" | "DESCRIPTIVE")} [mode="DESCRIPTIVE"] * @returns {Promise} * * * NUMERICAL MODE will use names like CF0001 for configurationId , DS0001,DS0002,DS0003... for data source ids and DP0001, DP0002... for dataPointIds * * DESCRIPTIVE MODE will use names like CF-assetName for configurationId , DS-aspectName... for data source ids and DP-variableName for data PointIds (default) * @memberOf MindConnectAgent */ public async GenerateDataSourceConfiguration( assetTypeId: string, mode: "NUMERICAL" | "DESCRIPTIVE" = "DESCRIPTIVE" ): Promise ``` ## GenerateMappings ```javascript /** * Generate automatically the mappings for the specified target assetid * * !Important! this only works if you have created the data source coniguration automatically * * @example * config = await agent.GenerateDataSourceConfiguration("castidev.Engine"); * await agent.PutDataSourceConfiguration(config); * const mappings = await agent.GenerateMappings(targetassetId); * await agent.PutDataMappings (mappings); * * @param {string} targetAssetId * @returns {Mapping[]} * * @memberOf MindConnectAgent */ public GenerateMappings(targetAssetId: string): Mapping[] ``` ## ConfigureAgentForAssetId ```javascript /** * This method can automatically create all necessary configurations and mappings for selected target asset id. * * * This method will automatically create all necessary configurations and mappings to start sending the data * * to an asset with selected assetid in Insights Hub * * @param {string} targetAssetId * @param {("NUMERICAL" | "DESCRIPTIVE")} mode * @param {boolean} [overwrite=true] ignore eTag will overwrite mappings and data source configuration * * * NUMERICAL MODE will use names like CF0001 for configurationId , DS0001,DS0002,DS0003... for data source ids and DP0001, DP0002... for dataPointIds * * DESCRIPTIVE MODE will use names like CF-assetName for configurationId , DS-aspectName... for data source ids and DP-variableName for data PointIds (default) * @memberOf MindConnectAgent */ public async ConfigureAgentForAssetId( targetAssetId: string, mode: "NUMERICAL" | "DESCRIPTIVE" = "DESCRIPTIVE", overwrite: boolean = true ) ``` ## SDK ```javascript /** * Insights Hub SDK using agent authentication * * ! important: not all APIs can be called with agent credentials, however Insights Hub is currently working on making this possible. * * * Here is a list of some APIs which you can use: * * * AssetManagementClient (Read Methods) * * MindConnectApiClient * * @returns {MindSphereSdk} * * @memberOf MindConnectAgent */ public Sdk(): MindSphereSdk ``` ## GetValidator ```javascript /** * Ajv Validator (@see https://github.com/ajv-validator/ajv) for the data points. Validates if the data points array is only * containing dataPointIds which are configured in the agent configuration. * * @returns {ajv.ValidateFunction} * * @memberOf MindConnectAgent */ public GetValidator(): ajv.ValidateFunction ``` ## GetEventValidator ```javascript /** * * Ajv Validator (@see https://github.com/ajv-validator/ajv) for the events. Validates the syntax of the events. * * @returns {ajv.ValidateFunction} * * @memberOf MindConnectAgent */ public GetEventValidator(): ajv.ValidateFunction ``` ## GetMindConnectConfiguration ```javascript /** * Get local configuration from the agent state storage. * * * This is a local agent state storage setting only. API is not called. * * @returns {IMindConnectConfiguration} * * @memberOf MindConnectAgent */ public GetMindConnectConfiguration(): IMindConnectConfiguration ``` # MindConnect-NodeJS - Agent Development - Retrying the operations ## Making sure that the data arrives also with an unreliable internet connection You can wrap all asynchronous object calls into the retry function which will automatically retry the operation for n times before throwing an exception. ```javascript import { MindConnectAgent, Insights HubStandardEvent, retry, TimeStampedDataPoint, } from "@mindconnect/mindconnect-nodejs"; // if you want to be more resillient you can wrap every async method // in the retry function which will try to retry several // times before throwing an exception // onboarding over flaky connection await retry(5, () => agent.OnBoard()); // bulk upload with 5 retries const bulk: TimeStampedDataPoint[] = [ { timestamp: "2018-08-23T18:38:02.135Z", values: [ { dataPointId: "DP-Temperature", qualityCode: "0", value: "10" }, { dataPointId: "DP-Pressure", qualityCode: "0", value: "10" }, ], }, { timestamp: "2018-08-23T19:38:02.135Z", values: [ { dataPointId: "DP-Temperature", qualityCode: "0", value: "10" }, { dataPointId: "DP-Pressure", qualityCode: "0", value: "10" }, ], }, ]; await retry(5, () => agent.BulkPostData(bulk)); ``` ## Uploading large files with multipart upload For the large files upload you would typically want to retry only the upload operations of the chunks and not the whole multipart upload once an error occurs. Therefore the UploadFile method offers the possibility to retry only the chunked upload: ```javascript // the upload-file can be a multipart operation and therefore can // be configured to retry the upload of the chunks instead // of the upload of the whole file. // if you don't specify the type, // the mimetype is automatically determined by the library await agent.UploadFile(agent.ClientId(), "custom/mindsphere/path/package.json", "package.json", { retry: RETRYTIMES, description: "File uploaded with MindConnect-NodeJS Library", chunk: true, // the chunk parameter activates multipart upload }); ``` # Mindconnect-NodeJS - Agent Development - Starter Projects ## Introduction The easiest way to start the development is to use the CLI to generate the starter projects ## Prerequisites You will need the [installed CLI](../cli/setting-up-the-cli.html). ## Step 1 - Create the project with help of the CLI ### Example Use the CLI to setup a starter project called my-agent: ```bash mdsp createts --dir my-agent ``` This will provide following output: ```text Starter project in my-agent has been created. Please run npm install in my-agent directory to install dependecies. ``` ```bash # after this we will install the required dependencies cd my-agent npm install ``` After this create the agentconfig.json with help of the CLI: ```bash mdsp create-agent --passkey{your passkey} --config agentconfig.json ``` This will produce following output ```text Configure your agent at: https://{yourtenant}-assetmanager.eu1.mindsphere.io/entity/{agentid}/plugin/uipluginassetmanagermclib ``` ## Step 2 - Create the assets in Insights Hub Create aspect types, asset type and asset in InsightsHub. Remember your asset id. ## Step 3 - Configure the agent You can use the new automatic creation of DataSourceConfiguration and DataSourceMappings to configure your agent. ```javascript import { DataPointValue, MindConnectAgent, MindSphereStandardEvent, retry, TimeStampedDataPoint, } from "@mindconnect/mindconnect-nodejs"; (async function () { const sleep = (ms: any) => new Promise((resolve) => setTimeout(resolve, ms)); const configuration = require("../../agentconfig.json"); const agent = new MindConnectAgent(configuration); const log = (text: any) => { console.log(`[${new Date().toISOString()}] ${text.toString()}`); }; const RETRYTIMES = 5; // retry the operation before giving up and throwing exception // onboarding the agent // Check in the local agent state storage if agent is onboarded. if (!agent.IsOnBoarded()) { // wrapping the call in the retry function makes the agent a bit more resillient // if you don't want to retry the operations you can always just call await agent.OnBoard(); instead. await retry(RETRYTIMES, () => agent.OnBoard()); log("Agent onboarded"); } // you can use agent.Sdk().GetAssetManagementClient() to get the asset id and asset type from Insights Hub. const targetAssetId = "1234567...."; const targetAssetType = `${agent.GetTenant()}.Engine`; // create data sourceconfiguration and mappings // this generates a data source configuration from an asset type const config = await agent.GenerateDataSourceConfiguration(targetAssetType); // create/overwrite the data source configuration await agent.PutDataSourceConfiguration(config); // create mappings for asset id const mappings = await agent.GenerateMappings(targetAssetId); // store mappings in InsightsHub await agent.PutDataMappings(mappings); // instead of creating the data source configuration and mappings separately // you can also just use the method below which takes care of everything // this is only used for 1:1 asset -> agent mappings // the method above can also map the data source configuration to multiple assets // just call GenerateMappings with different asset ids await agent.ConfigureAgentForAssetId(targetAssetId); // Check in the local agent state storage if agent has data source configuration. if (!agent.HasDataSourceConfiguration()) { await retry(RETRYTIMES, () => agent.GetDataSourceConfiguration()); log("Configuration aquired"); } for (let index = 0; index < 5; index++) { try { log(`Iteration : ${index}`); // if you have configred the data points in the Insights Hub UI you will have to use the long integer values instead of descriptive dataPointIds. const values: DataPointValue[] = [ { dataPointId: "DP-Temperature", qualityCode: "0", value: (Math.sin(index) * (20 + (index % 2)) + 25).toString(), }, { dataPointId: "DP-Pressure", qualityCode: "0", value: (Math.cos(index) * (20 + (index % 25)) + 25).toString(), }, { dataPointId: "DP-Humidity", qualityCode: "0", value: ((index + 30) % 100).toString(), }, { dataPointId: "DP-Acceleration", qualityCode: "0", value: (1000.0 + index).toString(), }, { dataPointId: "DP-Frequency", qualityCode: "0", value: (60.0 + index * 0.1).toString(), }, { dataPointId: "DP-Displacement", qualityCode: "0", value: (index % 10).toString(), }, { dataPointId: "DP-Velocity", qualityCode: "0", value: (50.0 + index).toString(), }, ]; // same like above, you can also just call await agent.PostData(values) if you don't want to retry the operation // this is how to send the data with specific timestamp // await agent.PostData(values, new Date(Date.now() - 86400 * 1000)); await retry(RETRYTIMES, () => agent.PostData(values)); log("Data posted"); await sleep(1000); const event: MindSphereStandardEvent = { entityId: agent.ClientId(), // use assetid if you want to send event somewhere else :) sourceType: "Event", sourceId: "application", source: "Meowz", severity: 20, // 0-99 : 20:error, 30:warning, 40: information timestamp: new Date().toISOString(), description: "Test", }; // send event with current timestamp; you can also just call agent.PostEvent(event) if you don't want to retry the operation await retry(RETRYTIMES, () => agent.PostEvent(event)); log("event posted"); await sleep(1000); // upload file // the upload-file can be a multipart operation and therefore can be configured to // retry the upload of the chunks instead the upload of the whole file. // if you don't specify the type , the mimetype is automatically determined by the library await agent.UploadFile( agent.ClientId(), "custom/mindsphere/path/package.json", "package.json", { retry: RETRYTIMES, description: "File uploaded with MindConnect-NodeJS Library", chunk: true, // the chunk parameter activates multipart upload } ); log("file uploaded"); await sleep(1000); const yesterday = new Date(); yesterday.setDate(yesterday.getDate() - 1); const bulk: TimeStampedDataPoint[] = [ { timestamp: yesterday.toISOString(), values: [ { dataPointId: "DP-Temperature", qualityCode: "0", value: "10" }, { dataPointId: "DP-Pressure", qualityCode: "0", value: "10" }, ], }, { timestamp: new Date().toISOString(), values: [ { dataPointId: "DP-Temperature", qualityCode: "0", value: "10" }, { dataPointId: "DP-Pressure", qualityCode: "0", value: "10" }, ], }, ]; await retry(RETRYTIMES, () => agent.BulkPostData(bulk)); log("bulk data uploaded"); await sleep(1000); } catch (err) { // add proper error handling (e.g. store data somewhere, retry later etc. ) console.error(err); } } })(); ``` ## Start the agent ```bash npm start ``` This should produce the output like this: ```text > mc-agent-ts@1.0.0 start > tsc && node index.js [2019-05-17T15:41:11.375Z] Iteration : 0 [2019-05-17T15:41:13.495Z] Agent onboarded [2019-05-17T15:41:14.398Z] Configuration aquired [2019-05-17T15:41:14.958Z] Data posted [2019-05-17T15:41:16.409Z] event posted [2019-05-17T15:41:18.285Z] bulk data uploaded [2019-05-17T15:41:19.288Z] Iteration : 1 [2019-05-17T15:41:19.716Z] Data posted [2019-05-17T15:41:21.325Z] event posted [2019-05-17T15:41:23.445Z] bulk data uploaded [2019-05-17T15:41:24.447Z] Iteration : 2 [2019-05-17T15:41:24.886Z] Data posted [2019-05-17T15:41:26.289Z] event posted [2019-05-17T15:41:30.744Z] bulk data uploaded ``` # MindConnect-NodeJS - CLI ## Introduction The library comes with a command line interface which can be used to upload timeseries, files and create events in the InsightsHub. It can also be used to create starter projects, list, create and delete assets, aspect and asset types, manage users and groups, create data lake permissions etc. ### Installation via `npm` package manager You will need [node.js](https://nodejs.org/) installed on your computer as prerequisite. ```bash npm install -g @mindconnect/mindconnect-nodejs ``` See also [getting started](setting-up-the-cli.html) section of this documentation ### Binary Releases The library can also be downloaded as executable from the [GitHub release page](https://github.com/mindsphere/mindconnect-nodejs/releases). Download the version for your system and place it in folder which is in your PATH. - `mdsp.exe` for windows - `mdsp-macos` for macOS - `mdsp-linux` for Linux Linux, macOS: Rename the file to `mdsp` and make sure that the file is marked as executable (`chmod +x`). ## Agent Commands | Command | Description | | ------------------------------------------------------------------------ | --------------------------------------------------------------------------------- | | [`mdsp onboard`](markdown-help/onboard.html) | onboard the agent with configuration stored in the config file | | [`mdsp configure-agent`](markdown-help/configure-agent.html) | create data source configuration and mappings (optional: passkey) \* | | [`mdsp agent-token`](markdown-help/agent-token.html) | displays the agent token for use in other tools (e.g. postman) | | [`mdsp upload-timeseries`](markdown-help/upload-timeseries.html) | parse .csv file with timeseriesdata and upload the timeseries data to InsightsHub | | [`mdsp upload-file`](markdown-help/upload-file.html) | upload the file to the InsightsHub file service (optional: passkey) \* | | [`mdsp create-event`](markdown-help/create-event.html) | create an event in the InsightsHub (optional: passkey) \* | | [`mdsp agent-status`](markdown-help/agent-status.html) | displays the agent status and agent onboarding status \* | | [`mdsp create-agent`](markdown-help/create-agent.html) | create an agent in the InsightsHub \* | | [`mdsp offboard-agent`](markdown-help/offboard-agent.html) | offboards the agent in the InsightsHub \* | | [`mdsp renew-agent`](markdown-help/renew-agent.html) | renews the agent secrets \* | | [`mdsp register-diagnostic`](markdown-help/register-diagnostic.html) | register agent for diagnostic \* | | [`mdsp get-diagnostic`](markdown-help/get-diagnostic.html) | get diagnostic information \* | | [`mdsp unregister-diagnostic`](markdown-help/unregister-diagnostic.html) | unregister agent from diagnostic \* | ## IoT Bulk Commands | Command | Description | | -------------------------------------------------------- | -------------------------------------------------------------------- | | [`mdsp prepare-bulk`](markdown-help/prepare-bulk.html) | creates a template directory for timeseries (bulk) upload \* | | [`mdsp run-bulk`](markdown-help/run-bulk.html) | runs the timeseries (bulk) upload job from directory \* | | [`mdsp check-bulk`](markdown-help/check-bulk.html) | checks the progress of the upload jobs from directory \* | | [`mdsp download-bulk`](markdown-help/download-bulk.html) | download the timeseries data in bulk from InsightsHub \* | | [`mdsp delivery-jobs`](markdown-help/delivery-jobs.html) | manage mqtt delivery jobs to publish MQTT commands to the clients \* | ## Assets, Files and Event Handling Commands | Command | Description | | -------------------------------------------------------- | ----------------------------------------- | | [`mdsp asset-lock`](markdown-help/asset-lock.html) | lock/unlock asset model modifications \* | | [`mdsp asset-info`](markdown-help/asset-info.html) | get infos about asset \* | | [`mdsp assets`](markdown-help/assets.html) | list, create or delete assets \* | | [`mdsp asset-types`](markdown-help/asset-types.html) | list, create or delete asset types \* | | [`mdsp aspects`](markdown-help/aspects.html) | list, create or delete aspects \* | | [`mdsp event-types`](markdown-help/event-types.html) | list, create or delete event types \* | | [`mdsp events`](markdown-help/events.html) | list, create or delete events \* | | [`mdsp events-bulk`](markdown-help/events-bulk.html) | download or delete the events in bulk \* | | [`mdsp aggregates`](markdown-help/aggregates.html) | list timeseries aggregates \* | | [`mdsp timeseries`](markdown-help/timeseries.html) | list timeseries \* | | [`mdsp notifications`](markdown-help/notifications.html) | send email, sms and push notifications \* | ## InsightsHub Open Edge Device Management Commands | Command | Description | | ------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------- | | [`mdsp oe-device-types`](markdown-help/oe-device-types.html) | list, create or delete device types (open edge) \* | | [`mdsp oe-devices`](markdown-help/oe-devices.html) | list, create or delete (open edge) devices \* | | [`mdsp oe-device-status`](markdown-help/oe-device-status.html) | list, get, or update (open edge) device status information \* | | [`mdsp oe-app-inst`](markdown-help/oe-app-inst.html) | list, create, configure or delete app instance (open edge) \* | | [`mdsp oe-app-deploy`](markdown-help/oe-app-deploy.html) | list, create, update app installation task(s) (open edge) \* | | [`mdsp oe-deploy-workflow`](markdown-help/oe-deploy-workflow.html) | list, create/instantiate, update or delete/cancel workflow deployment model or instance(s) (open edge) \* | | [`mdsp oe-firm-deploy`](markdown-help/oe-firm-deploy.html) | list, create, update firmware deployment task(s) (open edge) \* | ## Tenant Management Commands | Command | Description | | -------------------------------------------------- | ------------------------------------------------------- | | [`mdsp tenant`](markdown-help/tenant.html) | create or delete tenant legal configuration and logo \* | | [`mdsp subtenants`](markdown-help/subtenants.html) | list, create or delete subtenants \* | ## Deprecated Commands | Command | Description | | -------------------------------------------------------- | -------------------------------------------------- | | [`mdsp list-assets`](markdown-help/list-assets.html) | list assets in the tenant \* | | [`mdsp delete-asset`](markdown-help/delete-asset.html) | delete asset with id from InsightsHub \* | | [`mdsp list-files`](markdown-help/list-files.html) | list files stored with the asset \* | | [`mdsp download-file`](markdown-help/download-file.html) | download the file from InsightsHub file service \* | | [`mdsp delete-file`](markdown-help/delete-file.html) | delete the file from InsightsHub file service \* | ## Identity and Access Management Commands | Command | Description | | -------------------------------------------------------------------- | ---------------------------------------------------------------------- | | [`mdsp identity-management`](markdown-help/identity-management.html) | manage InsightsHub users, groups and roles \* | | [`mdsp policy`](markdown-help/policy.html) | list, create or delete policies \* | | [`mdsp service-credentials`](markdown-help/service-credentials.html) | provide login for commands which require technical user credentials \* | | [`mdsp service-token`](markdown-help/service-token.html) | displays the service token for use in other tools (e.g. postman) \* | ## Integrated Data Lake and Semantic Data Interconnect Commands | Command | Description | | -------------------------------------------------------------------- | ---------------------------------------------------------------- | | [`mdsp data-lake`](markdown-help/data-lake.html) | manage data lake, data lake access permissions and STS tokens \* | | [`mdsp sdi-data-lakes`](markdown-help/sdi-data-lakes.html) | manage data lakes for SDI \* | | [`mdsp sdi-data-registries`](markdown-help/sdi-data-registries.html) | manage data registries for SDI \* | | [`mdsp sdi-iot-registries`](markdown-help/sdi-iot-registries.html) | manage iot data registries for SDI \* | | [`mdsp sdi-data-types`](markdown-help/sdi-data-types.html) | manage data types for SDI \* | | [`mdsp sdi-file-upload`](markdown-help/sdi-file-upload.html) | upload file to SDI \* | | [`mdsp sdi-ingest-jobs`](markdown-help/sdi-ingest-jobs.html) | manage ingest jobs for SDI \* | | [`mdsp sdi-search-schemas`](markdown-help/sdi-search-schemas.html) | search SDI schemas \* | | [`mdsp sdi-data-queries`](markdown-help/sdi-data-queries.html) | manage data queries for SDI \* | | [`mdsp sdi-execution-jobs`](markdown-help/sdi-execution-jobs.html) | manage data execution jobs for SDI \* | | [`mdsp sdi-ontologies`](markdown-help/sdi-ontologies.html) | manage ontologies for SDI \* | | [`mdsp sdi-ontology-jobs`](markdown-help/sdi-ontology-jobs.html) | manage ontology jobs for SDI \* | ## Mobile Apps, Message Broker, Workorder Commands | Command | Description | | ---------------------------------------------------------------------- | --------------------------------------------------- | | [`mdsp mobile-apps`](markdown-help/mobile-apps.html) | list, create or delete mobile apps \* | | [`mdsp mobile-app-instances`](markdown-help/mobile-app-instances.html) | list, create or delete mobile app instances \* | | [`mdsp message-broker`](markdown-help/message-broker.html) | manage message broker subscriptions and webhooks \* | | [`mdsp cases`](markdown-help/cases.html) | list cases \* | ## Data Analytics Commands | Command | Description | | -------------------------------------------------------- | ------------------------------------------------------------------------ | | [`mdsp models`](markdown-help/models.html) | list, create or delete analytic models \* | | [`mdsp jobs`](markdown-help/jobs.html) | list, create or stop jobs \* | | [`mdsp schedules`](markdown-help/schedules.html) | list, create, start, stop or delete job schedules \* | | [`mdsp data-exchange`](markdown-help/data-exchange.html) | list, upload, download and manage data exchange files and directories \* | ## Developer Support Commands | Command | Description | | ---------------------------------------------------------- | ------------------------------------------------------------ | | [`mdsp dev-proxy`](markdown-help/dev-proxy.html) | starts InsightsHub development proxy & (optional passkey) \* | | [`mdsp mqtt-createjwt`](markdown-help/mqtt-createjwt.html) | creates a signed token for opcua pub sub authentication # | | [`mdsp starter-ts`](markdown-help/starter-ts.html) | creates a starter project in typescript # | | [`mdsp starter-js`](markdown-help/starter-js.html) | creates a starter project in javascript # | | [`mdsp markdown-help`](markdown-help/markdown-help.html) | generates folder with markdown help commands for the CLI\* | # MindConnect-NodeJS - Development Proxy The CLI comes with a development proxy which can be used to kickstart your development. It provides an endpoint at your local machine at which will authenticate all requests using either one of: - [SESSION and XSRF-TOKEN cookie](https://developer.mindsphere.io/howto/howto-local-development.html#generate-user-credentials) - **Recommended** - [Application Credentials](https://documentation.mindsphere.io/resources/html/developer-cockpit/en-US/124342231819.html) - **Recommended** - [Service Credentials](https://developer.mindsphere.io/howto/howto-selfhosted-api-access.html#creating-service-credentials) ## Developing with SESSION and XSRF-TOKEN Cookie First you should deploy an application to your tenant and configure its user rights. (any frontend app will do, including just an empty index.html. The important part is that you have configured the Insights Hub API scopes for your application). After that navigate to your application and create the MDSP_HOST, MDSP_SESSION and MDSP_XSRF_TOKEN like this (example bash shell, see [setting up the cli](setting-up-the-cli.html) for PowerShell and CMD examples ```bash export MDSP_HOST="castidev-sdk-castidev.eu1.mindsphere.io" export MDSP_SESSION="NzBi...Zl" export MDSP_XSRF_TOKEN="fed5edc5-...2565238c114" ``` After that the proxy will authorize all requests to Insights Hub as if the app would be deployed and if you would be logged in. ```bash mc dev-proxy ``` ## Developing with Application Credentials First you should deploy an application to your tenant as before. After that navigate to AppCredentials section in your developer cockpit and issue app credentials for your application. Configure the CLI using the ```bash mc service-credentials ``` command and select your application if you have multiple credentials configured. Start the proxy using the `-mode credentials` switch. ```bash mc dev-proxy --mode credentials --passkey ``` After that the proxy will authorize all requests to Insights Hub with selected Application Credentials. ## Development Proxy options Run `mc dev-proxy --help` for options: ```text Usage: mc dev-proxy|px [options] starts development proxy (optional passkey) * Options: -m, --mode [credentials|session] service/app credentials authentication of session authentication (default: "session") -o, --port port for web server (default: "7707") -r, --norewrite don't rewrite hal+json urls -w, --nowarn don't warn for missing headers -d, --dontkeepalive don't keep the session alive -v, --verbose verbose output -s, --session borrowed SESSION cookie from brower -x, --xsrftoken borrowed XSRF-TOKEN cookie from browser -h, --host the address where SESSION and XSRF-TOKEN have been borrowed from -t, --timeout keep alive timeout in seconds (default: "60") -k, --passkey passkey --help display help for command Examples: mc dev-proxy runs on default port (7707) using cookies mc dev-proxy --port 7777 --passkey passkey runs on port 7777 using app/service credentials Configuration: - create environment variables: MDSP_HOST, MDSP_SESSION and MDSP_XSRF_TOKEN using borrowed cookies ``` # MindConnect-NodeJS - Setting up the CLI The first step is to configure the CLI. For this, you will need application credentials (or deprecated service credentials) from your developer cockpit or SESSION and XSRF-TOKEN from the application you have been developing. - [Application Credentials](https://documentation.mindsphere.io/resources/html/developer-cockpit/en-US/124342231819.html) - **Recommended** - [Service Credentials](https://developer.mindsphere.io/howto/howto-selfhosted-api-access.html#creating-service-credentials) (deprecated) - [SESSION and XSRF-TOKEN cookie](https://developer.mindsphere.io/howto/howto-local-development.html#generate-user-credentials) **Important:** the service credentials (which are now deprecated) could only be acquired by emailing Insights Hub Support. They are not connected to the *Cloud Foundry Service Credentials* from Insights Hub settings app. ## Application Credentials / Service Credentials Configuration First start the credentials configuration. This will start a web server on your local computer where you can enter the credentials. ```bash # run mc service-credentials --help for full information $ mc service-credentials navigate to http://localhost:4994 to configure the CLI press CTRL + C to exit ``` Navigate to to configure the CLI. The image below shows the dialog for adding new credentials (press on the + sign in the upper left corner) You can get the application credentials from your developer or operator cockpit in Insights Hub. (if you don't have any application you can register a dummy one just for CLI) Once configured you can press CTRL + C to stop the configuration server and start using the CLI. Remember the passkey you have created ,as you will be using it with almost all CLI commands. ## Passkey as environment variable If you don't want to enter your passkey all the time you can also set it up as Environment Variable \`MDSP_PASSKEY' This will save you the typing of the `--passkey` option most of the time. Remember that this is not as secure as typing it in. Use with care. Bash: ```bash export MDSP_PASSKEY="my.complex.passkey" ``` Windows CMD ```text set "MDSP_PASSKEY=my.complex.passkey" ``` Windows PowerShell ```powershell $Env:MDSP_PASSKEY="my.complex.passkey" ``` The results of the commands will be colored in magenta if you are using Application of Service Credentials. ## Session Cookie - XSRF-Token configuration You can also use the SESSION Cookie and XSRF-TOKEN from the application you are developing in the CLI. Just configure following environment variables. ```bash export MDSP_HOST="mytenant-myapp.eu1.mindsphere.io" export MDSP_SESSION="ZDcyMWVkNjMtYXXXXXXXlkYmQtODYxZDljZjIzOGI1" export MDSP_XSRF_TOKEN="33771ee2-9650-XXXX-ab73-10f52cad12bf" ``` Windows CMD ```bash set "MDSP_HOST=mytenant-myapp.eu1.mindsphere.io" set "MDSP_SESSION=ZDcyMWVkNjMtYXXXXXXXlkYmQtODYxZDljZjIzOGI1" set "MDSP_XSRF_TOKEN=33771ee2-9650-XXXX-ab73-10f52cad12bf" ``` Windows PowerShell ```bash $Env:MDSP_HOST="mytenant-myapp.eu1.mindsphere.io"; $Env:MDSP_SESSION="ZDcyMWVkNjMtYXXXXXXXlkYmQtODYxZDljZjIzOGI1"; $Env:MDSP_XSRF_TOKEN="33771ee2-9650-XXXX-ab73-10f52cad12bf"; ``` The results of the commands will be colored yellow if you are using SESSION / XSRF-TOKEN type of authorization. The use of Service or Application credentials always takes precedence over Session Cookie / XSRF-TOKEN Authentication. ## Authentication Helper You can use the [Mindsphere Auth Helper](../../mindsphere-auth-helper/index.html) Chrome extension ( [chrome web store](https://chrome.google.com/webstore/detail/mindsphere-authentication/licndiiilobojikmhmmcgdbpmnmdeoee) ) to simplify the copying of the authentication cookies. The extension already provides the cookies in the proper format for the CLI so that you don't have to craft the MDSP_HOST, MDSP_SESSION and MDSP_XSRF_TOKEN variable manually. # mdsp agent-status Command Syntax: ```bash mdsp agent-status ``` Help: ```bash mdsp agent-status --help ``` Alternative form: ```bash mc agent-status ``` (The CLI was using `mc` as default command name in older versions) ## Description displays the agent status and agent onboarding status \* ## Usage Parameter list: ```text Usage: mdsp agent-status|as [options] displays the agent status and agent onboarding status * Options: -c, --config config file with agent configuration -a, --agentid agentid -k, --passkey passkey -r, --cert [privatekey] required for agents with RSA_3072 profile. create with: openssl genrsa -out private.key 3072 -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp agent-status` command: ```text Examples: mdsp agent-status uses default agentconfig.json mdsp agent-status --config agent.json uses specified configuration file mdsp agent-status --cert private.key uses specified key for RSA_3072 profile mdsp agent-status --passkey mypasskey displays also the online agent information mdsp agent-status --passkey mypasskey --verbose displays additionally the mappings and configuration ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp agent-token Command Syntax: ```bash mdsp agent-token ``` Help: ```bash mdsp agent-token --help ``` Alternative form: ```bash mc agent-token ``` (The CLI was using `mc` as default command name in older versions) ## Description displays the agent token for use in other tools (e.g. postman) ## Usage Parameter list: ```text Usage: mdsp agent-token|atk [options] displays the agent token for use in other tools (e.g. postman) Options: -c, --config config file with agent configuration (default: "agentconfig.json") -k, --passkey passkey -r, --cert [privatekey] required for agents with RSA_3072 profile. create with: openssl genrsa -out private.key 3072 -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp agent-token` command: ```text Examples: mdsp agent-token uses default agentconfig.json mdsp agent-token --config agent.json uses specified configuration file mdsp agent-token --cert private.key uses specified key for RSA_3072 profile mdsp agent-token --verbose displays encoded and decoded version of the token ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp aggregates Command Syntax: ```bash mdsp aggregates ``` Help: ```bash mdsp aggregates --help ``` Alternative form: ```bash mc aggregates ``` (The CLI was using `mc` as default command name in older versions) ## Description list timeseries aggregates \* ## Usage Parameter list: ```text Usage: mdsp aggregates|ag [options] list timeseries aggregates * Options: -i, --assetid Insights Hub asset id -n, --aspectname Insights Hub aspect name -f, --from begining of the time range to read (default: "2022-11-05T01:51:16.065Z") -t, --to end of the time range to read -r, --intervalvalue interval duration for the aggregates in interval units -u, --intervalunit interval duration unit [minute |hour |day |week | month] -s, --select comma separated list of variable names -d, --download [download] download aggregates to specified file -a, --all show all aggregates not just average, min, max, sum and sd -l, --local use localtime in aggregate list -c, --count number of aggregates in response -h, --formatted write JSON strings with indentation -p, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp aggregates` command: ```text Examples: mdsp aggregates --asssetid 1234567..ef --aspectname Environment list recent aggregates for aspect Environment mdsp aggregates --asssetid 1234567..ef --aspectname Environment --select Temperature list recent temperature aggregates mdsp aggregates --asssetid 1234567..ef --aspectname Environment --select Temperature --all list all recent temperature aggregates mdsp aggregates --asssetid 1234567..ef --aspectname Environment --intervalunit hour --intervalvalue 2 list all recent temperatre aggregates over every 2 hours Important: see https://documentation.mindsphere.io/MindSphere/apis/iot-iottsaggregates/api-iottsaggregates-samples-v4.html for documentation about the aggregate parameters ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp aspects Command Syntax: ```bash mdsp aspects ``` Help: ```bash mdsp aspects --help ``` Alternative form: ```bash mc aspects ``` (The CLI was using `mc` as default command name in older versions) ## Description list, create or delete aspects \* ## Usage Parameter list: ```text Usage: mdsp aspects|asp [options] list, create or delete aspects * Options: -m, --mode [list|create|delete|convert|template|info] list | create | delete | convert | template | info (default: "list") -f, --file .mdsp.json file with aspect type definition -s, --schema JSON Schema -n, --aspect the aspect type name -p, --prefixflattened prefix variable names with previous name when flattening schema -t, --targetonly consider only variables which have target equal to aspect in schema -u, --untargeted consider only variables which don't have target in schema -l, --length default string length (default: "255") -b, --biglength default bigstring length (default: "5000") -i, --idonly list only ids -c, --includeshared include shared aspect types -k, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp aspects` command: ```text Examples: mdsp aspects --mode list list all aspect types mdsp aspects --mode list --aspect Environment list all aspect types which are named Environment mdsp aspects --mode template --aspect Environment create a template file (Enironment.aspect.mdsp.json) for aspect Environment mdsp aspects --mode create --file Environment.aspects.mdsp.json create aspect type Environment in Insights Hub mdsp aspects --mode convert --schema Environment.schema.json --aspect Environment create a template file for aspect type Environment from JSON schema mdsp aspects --mode convert --schema Environment.schema.json --aspect Environment --prefixflattened prefixes the variable names with parent object names (e.g. Environment_Temperature) mdsp aspects --mode convert --schema Environment.schema.json --aspect Environment --targetonly select only variables from json schema with target property equal to assettype ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp asset-info Command Syntax: ```bash mdsp asset-info ``` Help: ```bash mdsp asset-info --help ``` Alternative form: ```bash mc asset-info ``` (The CLI was using `mc` as default command name in older versions) ## Description get infos about asset \* ## Usage Parameter list: ```text Usage: mdsp asset-info|ai [options] get infos about asset * Options: -i, --assetid Insights Hub asset id -k, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp asset-info` command: ```text Examples: mdsp asset-info --assetid 123456...ef print out infos about asset with id 132456...ef ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp asset-lock Command Syntax: ```bash mdsp asset-lock ``` Help: ```bash mdsp asset-lock --help ``` Alternative form: ```bash mc asset-lock ``` (The CLI was using `mc` as default command name in older versions) ## Description lock/unlock asset model modifications \* ## Usage Parameter list: ```text Usage: mdsp asset-lock|lck [options] lock/unlock asset model modifications * Options: -m, --mode [info|lock|unlock] mode [info | lock | unlock] (default: "info") -k, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp asset-lock` command: ```text Examples: mdsp asset-lock --mode info print out the asset model lock state mdsp asset-lock --mode lock lock the asset model and disable modifications mdsp asset-lock --mode unlock unlock the asset model and enable modifications ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp asset-types Command Syntax: ```bash mdsp asset-types ``` Help: ```bash mdsp asset-types --help ``` Alternative form: ```bash mc asset-types ``` (The CLI was using `mc` as default command name in older versions) ## Description list, create or delete asset types \* ## Usage Parameter list: ```text Usage: mdsp asset-types|at [options] list, create or delete asset types * Options: -m, --mode [list|create|delete|template | info] list | create | delete | template | info (default: "list") -f, --file .mdsp.json file with asset type definition -i, --idonly list only ids -s, --schema JSON Schema -n, --assettype the asset type name -c, --includeshared include shared asset types -k, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp asset-types` command: ```text Examples: mdsp asset-types --mode list list all asset types mdsp asset-types --mode list --assettype Pump list all asset types which are named Pump mdsp asset-types --mode template --assettype Pump create a template file (Enironment.assettype.mdsp.json) for assettype Pump mdsp asset-types --mode create --file Pump.assettype.mdsp.json create asset type Pump in Insights Hub ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp assets Command Syntax: ```bash mdsp assets ``` Help: ```bash mdsp assets --help ``` Alternative form: ```bash mc assets ``` (The CLI was using `mc` as default command name in older versions) ## Description list, create or delete assets \* ## Usage Parameter list: ```text Usage: mdsp assets|ast [options] list, create or delete assets * Options: -m, --mode [list|create|delete|template|tree] mode [list | create | delete | template | tree] (default: "list") -f, --file .mdsp.json file with asset definition -n, --assetname assetname -p, --parentid parentid -e, --externalid externalid -i, --assetid Insights Hub asset id -t, --typeid typeid -d, --desc description (default: "created with Insights Hub CLI") -w, --twintype digital twin type [performance|simulation] -c, --includeshared include shared aspect types -k, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp assets` command: ```text Examples: mdsp assets --mode create --typeid core.basicarea --assetname MyArea creates an asset in Insights Hub of type basicarea mdsp assets --mode create --file MyPump.asset.mdsp.json creates an asset from specified file template mdsp assets --mode list lists all assets in Insights Hub mdsp assets --mode list --typeid mclib lists all assets in Insights Hub of type core.mclib mdsp assets --mode delete --assetid 1234567..ef deletes asset with specified id from Insights Hub mdsp assets --mode template --typeid .Pump --assetname MyPump creates a file template MyPump.asset.mdsp.json which can be use in create command ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp cases Command Syntax: ```bash mdsp cases ``` Help: ```bash mdsp cases --help ``` Alternative form: ```bash mc cases ``` (The CLI was using `mc` as default command name in older versions) ## Description list, create or delete cases \* ## Usage Parameter list: ```text Usage: mdsp cases|cs [options] list, create or delete cases * Options: -m, --mode [list|create|update|delete|template|info] list | create | update | delete | template | info (default: "list") -f, --file .mdsp.json file with case definition (default: "case.mdsp.json") -n, --case the case name -i, --handle the case handle -o, --overwrite overwrite template file if it already exists -k, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp cases` command: ```text Examples: mdsp cases --mode list list all cases mdsp cases --mode template --case create a template file for mdsp cases --mode create --file create case mdsp cases --mode update --file --handle update case mdsp cases --mode info --handle case info for specified handle mdsp cases --mode delete --handle delete case with specified handle ``` See [MindSphere API documentation](https://developer.siemens.com/insights-hub/overview.html) for more information about MindSphere APIs. # mdsp check-bulk Command Syntax: ```bash mdsp check-bulk ``` Help: ```bash mdsp check-bulk --help ``` Alternative form: ```bash mc check-bulk ``` (The CLI was using `mc` as default command name in older versions) ## Description checks the progress of the upload jobs from directory \* ## Usage Parameter list: ```text Usage: mdsp check-bulk|cb [options] checks the progress of the upload jobs from directory * Options: -d, --dir config file with agent configuration (default: "bulkupload") -y, --retry retry attempts before giving up (default: "3") -i, --timeseries use (deprecated) standard timeseries upload -k, --passkey passkey -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp check-bulk` command: ```text Examples: mdsp check-bulk displays job progress of bulkimport directory mdsp check-bulk --dir asset1 --verbose displays job progress of asset1 directory with verbose output ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp configure-agent Command Syntax: ```bash mdsp configure-agent ``` Help: ```bash mdsp configure-agent --help ``` Alternative form: ```bash mc configure-agent ``` (The CLI was using `mc` as default command name in older versions) ## Description create data source configuration and mappings (optional: passkey) \* ## Usage Parameter list: ```text Usage: mdsp configure-agent|co [options] create data source configuration and mappings (optional: passkey) * Options: -c, --config config file with agent configuration -m, --mode [mode] command mode [config|map|print|delete|test|func|template] (default: "config") -a, --agentid agentid -i, --assetid target assetid for mapping -t, --typeid asset type for configuration -l, --language [js|python|json] target language for conversion function (default: "js") -s, --timespan timespan between generated timestamps (in ms) (default: "1000") -c, --count Number of generated records (default: "10") -k, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp configure-agent` command: ```text Examples: mdsp configure-agent --config agent.json -assetid 1234567...89 configures agent for specified assetid mdsp configure-agent --config agent.json --mode print prints data source configuration and mappings mdsp configure-agent --agentid 12345..ef --typeid .Engine creates the data source configuration mdsp configure-agent --mode map --agentid 12345..ef --assetid 1234567 creates the mappings for assetid mdsp configure-agent --mode delete --agentid 12345..ef deletes the mappings for agentid mdsp configure-agent --config agent.json --mode test sends test data to Insights Hub mdsp configure-agent --mode template \ --typeid castidev.Pump --language python create mapping template and function in python ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp create-agent Command Syntax: ```bash mdsp create-agent ``` Help: ```bash mdsp create-agent --help ``` Alternative form: ```bash mc create-agent ``` (The CLI was using `mc` as default command name in older versions) ## Description create an agent in the Insights Hub \* ## Usage Parameter list: ```text Usage: mdsp create-agent|ca [options] create an agent in the Insights Hub * Options: -c, --config config file for agent configuration -r, --cert [privatekey] required for agents with RSA_3072 profile. create with: openssl genrsa -out private.key 3072 -n, --agentname agent name (default: "Agent1667699476184") -p, --parentid parent asset id -f, --profile security profile [SHARED_SECRET|RSA_3072] (default: "SHARED_SECRET") -k, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp create-agent` command: ```text Examples: mdsp create-agent --config agent.json --passkey passkey... create agent with default parameters ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp create-event Command Syntax: ```bash mdsp create-event ``` Help: ```bash mdsp create-event --help ``` Alternative form: ```bash mc create-event ``` (The CLI was using `mc` as default command name in older versions) ## Description create an event in the Insights Hub (optional: passkey) \* ## Usage Parameter list: ```text Usage: mdsp create-event|ce [options] create an event in the Insights Hub (optional: passkey) * Options: -c, --config config file with agent configuration -r, --cert [privatekey] required for agents with RSA_3072 profile. create with: openssl genrsa -out private.key 3072 -i, --assetid Insights Hub asset id (default: send event to the agent) -y, --sourceType Source Type (default: "MindConnect-Agent") -S, --sourceId Source Id (default: "Monikas-Air-2.fritz.box") -O, --source Source (default: "MindConnect-NodeJs CLI") -V, --severity Severity (20:Error, 30:Warning , 40:information) (default: "20") -d, --desc Event description (default: "CLI created event") -t, --timestamp Timestamp (default: "2022-11-06T01:51:16.184Z") -y, --retry retry attempts before giving up (default: "3") -p, --passkey passkey (optional, event creation uses service credentials *) -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp create-event` command: ```text Examples: mdsp create-event create error event with default values and current timestamp mdsp ce --desc Warning! --severity 30 create warning with description warning mdsp ce --desc "custom event" --i 123....4 create error event for asset with id 123....4 ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp data-exchange Command Syntax: ```bash mdsp data-exchange ``` Help: ```bash mdsp data-exchange --help ``` Alternative form: ```bash mc data-exchange ``` (The CLI was using `mc` as default command name in older versions) ## Description list, upload, download and manage data exchange files and directories \* ## Usage Parameter list: ```text Usage: mdsp data-exchange|dx [options] list, upload, download and manage data exchange files and directories * Options: -m, --mode [list|info|download|upload|rename|delete|renamedir|mkdir|rmdir] mode [list | info | download | upload | rename | renamedir | delete | mkdir | rmdir ] (default: "list") -f, --file file path -n, --dirname directory name for --mode mkdir command -w, --newname new file or directory name for --mode rename or renamedir command -d, --dirid directory id [private | public | ] (default: "public") -i, --fileid fileid -r, --recursive used for --mode rmdir command, deletes all content in directory -k, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp data-exchange` command: ```text Examples: mdsp data-exchange --mode list list all entries in public data exchange root mdsp data-exchange --mode list --dirid private list all entries in private data exchange root mdsp data-exchange --mode info --dirid get full info about the specified directory mdsp data-exchange --mode info --fileid get full info about the specified mdsp data-exchange --mode download --fileid download file with specified id mdsp data-exchange --mode upload --file --dirid upload file to specified directory mdsp data-exchange --mode rename --fileid --newname rename the specified file mdsp data-exchange --mode renamedir --dirid --newname rename the specified directory mdsp data-exchange --mode delete --fileid delete file with specified id mdsp data-exchange --mode rmdir --dirid delete directory with specified id mdsp data-exchange --mode rmdir --dirid --recursive delete directory with specified id recrusively ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp data-lake Command Syntax: ```bash mdsp data-lake ``` Help: ```bash mdsp data-lake --help ``` Alternative form: ```bash mc data-lake ``` (The CLI was using `mc` as default command name in older versions) ## Description manage data lake, data lake access permissions and STS tokens \* ## Usage Parameter list: ```text Usage: mdsp data-lake|dlk [options] manage data lake, data lake access permissions and STS tokens * Options: -m, --mode [...] mode: list | read | write | delete | readtoken | writetoken | uploadurl | downloadurl | upload | meta | subscriptions | subscribe | unsubscribe (default: "list") -f, --file file to upload -l, --shell [bash|ps|cmd] output format for STS token [bash | ps | cmd] (default: "bash") -p, --path path for read/write token or uploadUrl, downloadUrl, subscribe or unsubscribe comamand -t, --destination destination for subscribe comamand -i, --permissionid permission id for delete operation -s, --subtenantid subtenant id -b, --subscriptionid subscription id -d, --duration duration in seconds (default: "3600") -k, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp data-lake` command: ```text Examples: mdsp data-lake --mode list lists all configured permissions for data lake mdsp data-lake --mode write allow writing to the data lake mdsp data-lake --mode write --path data/ allow writing to the data lake from data/ folder mdsp data-lake --mode meta --path data/ get metadata for path mdsp data-lake --mode readtoken create AWS STS Token with read rights mdsp data-lake --mode writetoken create AWS STS Token with write rights from data lake root mdsp data-lake --mode writetoken --path data/ create AWS STS Token with write rights from data/ folder mdsp data-lake --mode delete --permissionid delete writing permission with selected permissionid mdsp data-lake --mode upload --file CHANGELOG.md --path uploads/CHANGELOG.md upload file to data lake mdsp data-lake --mode downloadurl --path uploads/CHANGELOG.md generate download url for the path mdsp data-lake --mode subscriptions list all data lake event subscriptions mdsp data-lake --mode subscribe --path --destination aws-sns:// subscribe an AWS SNS topic to folder changes mdsp data-lake --mode unsubscribe --subscriptionid delete event subscription Additional Information: Purchasing Data Lake: https://www.dex.siemens.com/mindsphere/applications/integrated-data-lake Data Lake APIs: https://developer.mindsphere.io/apis/iot-integrated-data-lake/api-integrated-data-lake-overview.html ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp delete-asset Command Syntax: ```bash mdsp delete-asset ``` Help: ```bash mdsp delete-asset --help ``` Alternative form: ```bash mc delete-asset ``` (The CLI was using `mc` as default command name in older versions) ## Description delete asset with id from Insights Hub \* ## Usage Parameter list: ```text Usage: mdsp delete-asset|da [options] delete asset with id from Insights Hub * Options: -i, --assetid Insights Hub asset id -k, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp delete-asset` command: ```text Examples: mdsp delete-asset --assetid 123456...ef delete asset with id 132456...ef ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp delete-file Command Syntax: ```bash mdsp delete-file ``` Help: ```bash mdsp delete-file --help ``` Alternative form: ```bash mc delete-file ``` (The CLI was using `mc` as default command name in older versions) ## Description delete the file from Insights Hub file service \* ## Usage Parameter list: ```text Usage: mdsp delete-file|de [options] delete the file from Insights Hub file service * Options: -f, --file file to delete from the file service -h, --filepath [filepath] file path in the Insights Hub (default: "") -i, --assetid Insights Hub asset id -p, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp delete-file` command: ```text Examples: mdsp delete-file -f CHANGELOG.md --assetid 5..f delete file CHANGELOG.md from specified asset mdsp delete-file --file CHANGELOG.md --assetid 5...f --filepath upload delete file upload/CHANGELOG.md from specified asset mdsp delete-file --file upload/CHANGELOG.md --assetid 5...f delete file upload/CHANGELOG.md from specified asset ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp delivery-jobs Command Syntax: ```bash mdsp delivery-jobs ``` Help: ```bash mdsp delivery-jobs --help ``` Alternative form: ```bash mc delivery-jobs ``` (The CLI was using `mc` as default command name in older versions) ## Description manage mqtt delivery jobs to publish MQTT commands to the clients \* ## Usage Parameter list: ```text Usage: mdsp delivery-jobs|dj [options] manage mqtt delivery jobs to publish MQTT commands to the clients * Options: -m, --mode [list|create|delete|template|info|commands|commandinfo] list | create | delete | template | info | commands | commandinfo (default: "list") -f, --file .mdsp.json file with job definition (default: "deliveryjob.mqtt.mdsp.json") -i, --jobid the job id -c, --commandid the command id -e, --name the name filter (contains) for list command -t, --filter the JSON filter as specified in the API documentation -o, --overwrite overwrite template file if it already exists -k, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp delivery-jobs` command: ```text Examples: mdsp delivery-jobs --mode list list all jobs mdsp delivery-jobs --mode list --name list all mqtt delivery jobs with the given name mdsp delivery-jobs --mode list --filter '{"createdAt": {"after": "2021-11-06T13:46:00Z"}}' list all mqtt delivery jobs created after the specified date mdsp delivery-jobs --mode template create template file for job creation mdsp delivery-jobs --mode create --file create job mdsp delivery-jobs --mode info --jobid get infos about the job mdsp delivery-jobs --mode delete --jobid delete job with job id mdsp delivery-jobs --mode commands --jobid list all commands for specified job id mdsp delivery-jobs --mode commandinfo --jobid --commandid get info for selected command ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp dev-proxy Command Syntax: ```bash mdsp dev-proxy ``` Help: ```bash mdsp dev-proxy --help ``` Alternative form: ```bash mc dev-proxy ``` (The CLI was using `mc` as default command name in older versions) ## Description starts Insights Hub development proxy & (optional passkey) \* ## Usage Parameter list: ```text Usage: mdsp dev-proxy|px [options] starts Insights Hub development proxy & (optional passkey) * Options: -m, --mode [credentials|session] service/app credentials authentication of session authentication (default: "session") -o, --port port for web server (default: "7707") -r, --norewrite don't rewrite hal+json urls -w, --nowarn don't warn for missing headers -d, --dontkeepalive don't keep the session alive -v, --verbose verbose output -s, --session borrowed SESSION cookie from brower -x, --xsrftoken borrowed XSRF-TOKEN cookie from browser -h, --host the address where SESSION and XSRF-TOKEN have been borrowed from -t, --timeout keep alive timeout in seconds (default: "60") -k, --passkey passkey --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp dev-proxy` command: ```text Examples: mdsp dev-proxy runs on default port (7707) using cookies mdsp dev-proxy --mode credentials --port 7777 --passkey $MDSP_PASSKEY runs on port 7777 using app/service credentials Configuration: - create environment variables: MDSP_HOST, MDSP_SESSION and MDSP_XSRF_TOKEN using borrowed cookies ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp download-bulk Command Syntax: ```bash mdsp download-bulk ``` Help: ```bash mdsp download-bulk --help ``` Alternative form: ```bash mc download-bulk ``` (The CLI was using `mc` as default command name in older versions) ## Description download the timeseries data in bulk from Insights Hub \* ## Usage Parameter list: ```text Usage: mdsp download-bulk|db [options] download the timeseries data in bulk from Insights Hub * Options: -d, --dir directory for the download (shouldn't exist) (default: "bulkdownload") -i, --assetid Insights Hub asset id -a, --aspectname aspectname -f, --from from date -t, --to to date -s, --size max entries per file (default: "200") -p, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp download-bulk` command: ```text Examples: mdsp download-bulk --assetid 12345..ef --from 12/10/2019 --to 12/16/2019 download timeseries from specified asset ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp download-file Command Syntax: ```bash mdsp download-file ``` Help: ```bash mdsp download-file --help ``` Alternative form: ```bash mc download-file ``` (The CLI was using `mc` as default command name in older versions) ## Description download the file from Insights Hub file service \* ## Usage Parameter list: ```text Usage: mdsp download-file|df [options] download the file from Insights Hub file service * Options: -f, --file file to download from the file service -h, --filepath [filepath] file path in the Insights Hub (default: "") -i, --assetid Insights Hub asset id -p, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp download-file` command: ```text Examples: mdsp download-file -f CHANGELOG.md --assetid 5..f download file CHANGELOG.md from specified asset mdsp download-file --file CHANGELOG.md --assetid 5...f --filepath upload download file upload/CHANGELOG.md from specified asset mdsp download-file --file upload/CHANGELOG.md --assetid 5...f download file upload/CHANGELOG.md from specified asset ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp event-types Command Syntax: ```bash mdsp event-types ``` Help: ```bash mdsp event-types --help ``` Alternative form: ```bash mc event-types ``` (The CLI was using `mc` as default command name in older versions) ## Description list, create or delete event types \* ## Usage Parameter list: ```text Usage: mdsp event-types|et [options] list, create or delete event types * Options: -m, --mode [list|create|delete|template|info] list | create | delete | template | info (default: "list") -f, --file .mdsp.json file with event type definition -i, --idonly list only ids -s, --schema JSON Schema -n, --eventtype the event type name -c, --includeshared include shared event types -g, --includeglobal include global event types -k, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp event-types` command: ```text Examples: mdsp event-types --mode list list all event types mdsp event-types --mode list --eventtype PumpEvent list all event types which are named PumpEvent mdsp event-types --mode template --eventtype PumpEvent create a template file (PumpEvent.eventtype.mdsp.json) for event type PumpEvent mdsp event-types --mode create --file PumpEvent.eventtype.mdsp.json create event type PumpEvent in Insights Hub ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp events-bulk Command Syntax: ```bash mdsp events-bulk ``` Help: ```bash mdsp events-bulk --help ``` Alternative form: ```bash mc events-bulk ``` (The CLI was using `mc` as default command name in older versions) ## Description download or delete the events in bulk \* ## Usage Parameter list: ```text Usage: mdsp events-bulk|dn [options] download or delete the events in bulk * Options: -m, --mode [download|template|delete|check] mode [download | template | delete | check] (default: "download") -d, --dir download folder (default: "eventdownload") -i, --assetid Insights Hub asset id -j, --jobid check deletion process of jobs with jobid -f, --filter [filter] JSON file with filter (see: https://developer.mindsphere.io/apis/advanced-eventmanagement/api-eventmanagement-best-practices.html) -s, --size max entries per file (default: "100") -t, --sort sort events (default: "timestamp,asc") -p, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp events-bulk` command: ```text Examples: mdsp events-bulk --mode download --asssetid 1234567..ef download events from specified asset mdsp events-bulk --mode download --dir newdir download last 7 days of events to folder mdsp events-bulk --mode template --assetid 1234576..ef create template file event.filter.mdsp.json mdsp events-bulk --mode download --filter event.filter.mdsp.json download events using configured filter mdsp events-bulk --mode delete --filter event.filter.mdsp.json delete events using configured filter mdsp events-bulk --mode check --jobid check the state of bulk deleting job ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp events Command Syntax: ```bash mdsp events ``` Help: ```bash mdsp events --help ``` Alternative form: ```bash mc events ``` (The CLI was using `mc` as default command name in older versions) ## Description list, create or delete events \* ## Usage Parameter list: ```text Usage: mdsp events|ev [options] list, create or delete events * Options: -m, --mode [list|create|delete|template|filtertemplate|info] list | create | delete | template | filtertemplate | info (default: "list") -f, --file .mdsp.json file with event definition -i, --assetid Insights Hub asset id -e, --eventid event id -f, --filter [filter] JSON file with filter (see: https://developer.mindsphere.io/apis/advanced-eventmanagement/api-eventmanagement-best-practices.html) -n, --eventtype the event type name -s, --size max. number of events to list (default: "100") -c, --includeshared include shared event types -g, --includeglobal include global event types -k, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp events` command: ```text Examples: mdsp events --mode list list last events (default:100) mdsp events --mode list --eventtype PumpEvent list last PumpEvents (default: 100) mdsp events --mode info --eventid get info about event with specified id mdsp events --mode delete --eventid delete event with specified id mdsp events --mode filtertemplate create filter template for --mode list command mdsp events --mode template --eventtype PumpEvent create a template file for event mdsp events --mode create --file PumpEvent.eventtype.mdsp.json create event ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp get-diagnostic Command Syntax: ```bash mdsp get-diagnostic ``` Help: ```bash mdsp get-diagnostic --help ``` Alternative form: ```bash mc get-diagnostic ``` (The CLI was using `mc` as default command name in older versions) ## Description get diagnostic information \* ## Usage Parameter list: ```text Usage: mdsp get-diagnostic|gd [options] get diagnostic information * Options: -c, --config config file with agent configuration -a, --agentid agent id -k, --passkey passkey -l, --all display all entries not just the last page -j, --json json output -t, --text text (raw) output -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp get-diagnostic` command: ```text Examples: mdsp gd -k mypasskey mdsp get-diagnostic --config someagent.json --passkey mypasskey mdsp get-diagnostic --passkey mypasskey --text --all > log.csv ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp identity-management Command Syntax: ```bash mdsp identity-management ``` Help: ```bash mdsp identity-management --help ``` Alternative form: ```bash mc identity-management ``` (The CLI was using `mc` as default command name in older versions) ## Description manage Insights Hub users, groups and roles \* ## Usage Parameter list: ```text Usage: mdsp identity-management|iam [options] manage Insights Hub users, groups and roles * Options: -m, --mode [list|create|assign|remove|delete|info] Mode can be list | create | assign | remove | delete | info (default: "list") -u, --user [user] user name -g, --group [group] user group -r, --role [role] user role -s, --subtenant subtenant for user -m, --membergroup [membergroup] member group -l, --memberrole [memberrole] member role -t, --meta include meta information (ids, login details etc.) -w, --raw don't automatically preceed group names with mdsp_usergroup or role with mdsp_customrole -k, --passkey passkey -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp identity-management` command: ```text Example: mdsp iam --mode list --user list all users mdsp iam --mode list --user [user] list all users which contain [user] in the login name mdsp iam --mode list --group list all groups mdsp iam --mode list --group [group] list all groups which contain [group] in the name mdsp iam --mode list --role list all roles mdsp iam --mode list --role [role] list all roles which contain [role] in the name mdsp iam --mode info --user [user] --meta get all infos about users which contain [user] in the login name mdsp iam --mode info --group [group] --meta get all infos about groups which contain [group] in the name mdsp iam --mode info --role [role] --meta get all infos about roles which contain [role] in the name mdsp iam --mode create|delete --user create or delete user mdsp iam --mode create|delete --group create or delete group mdsp iam --mode create|delete --role create or delete role mdsp iam --mode create --user --subtenant subtenant create user in specified subtenant mdsp iam --mode assign --user --group assign user to user group group mdsp iam --mode assign --user --role assign role to user user mdsp iam --mode assign --user --group assign user to user group group mdsp iam --mode assign --group --membergroup assign membergroup to user group group mdsp iam --mode assign --group --role assign role to user group group mdsp iam --mode assign --role --memberrole assign member role to custom role role mdsp iam --mode remove --user --group remove user from user group group mdsp iam --mode remove --user --role remove role from user user mdsp iam --mode remove --user --group remove user from user group group mdsp iam --mode remove --group --membergroup remove membergroup from user group group mdsp iam --mode remove --group --role remove role from user group group mdsp iam --mode remove --role --memberrole remove member role from custom role role ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp jobs Command Syntax: ```bash mdsp jobs ``` Help: ```bash mdsp jobs --help ``` Alternative form: ```bash mc jobs ``` (The CLI was using `mc` as default command name in older versions) ## Description list, create or stop jobs \* ## Usage Parameter list: ```text Usage: mdsp jobs|jb [options] list, create or stop jobs * Options: -m, --mode [list|create|stop|template|info] list | create | stop | template | info (default: "list") -f, --file .mdsp.json file with job definition (default: "jobmanager.job.mdsp.json") -i, --jobid the job id -e, --message the message filter (contains) for list command -s, --status the status filter (equals, e.g. STOPPED, FAILED...) for list command -d, --modelid the modelid filter (equals) for list command -o, --overwrite overwrite template file if it already exists -k, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp jobs` command: ```text Examples: mdsp jobs --mode list list all jobs mdsp jobs --mode list --status FAILED --message import list all jobs which failed on import mdsp jobs --mode list --modelid list all jobs for specified model mdsp jobs --mode list --modelid list all jobs for specified model mdsp jobs --mode list --modelid list all jobs for specified model mdsp jobs --mode template create template file for job creation mdsp jobs --mode create --file create job mdsp jobs --mode info --jobid get infos about the job mdsp jobs --mode stop --jobid stop job with job id ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp list-assets Command Syntax: ```bash mdsp list-assets ``` Help: ```bash mdsp list-assets --help ``` Alternative form: ```bash mc list-assets ``` (The CLI was using `mc` as default command name in older versions) ## Description list assets in the tenant \* ## Usage Parameter list: ```text Usage: mdsp list-assets|la [options] list assets in the tenant * Options: -f, --filter [filter] filter (see: https://developer.mindsphere.io/apis/advanced-assetmanagement/api-assetmanagement-references-filtering.html) -a, --assetname [name] search for assets with string [name] in asset name -t, --typeid [typeid] search for assets with string [typeid] in typeid -c, --includeshared include shared assets -k, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp list-assets` command: ```text Examples: mdsp list-assets --passkey mypasskey list all assets mdsp la --typeid core.mclib --assetname nodered --passkey mypasskey list all agents (assets of type core.mclib) with nodered in the name mdsp la --filter '{"name" : {"contains" : "Engine"}}' --passkey mypasskey list all assets where name contains string Engine ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp list-files Command Syntax: ```bash mdsp list-files ``` Help: ```bash mdsp list-files --help ``` Alternative form: ```bash mc list-files ``` (The CLI was using `mc` as default command name in older versions) ## Description list files stored with the asset \* ## Usage Parameter list: ```text Usage: mdsp list-files|ls [options] list files stored with the asset * Options: -i, --assetid Insights Hub asset id -f, --filter [filter] filter (see: https://developer.mindsphere.io/apis/iot-iotfile/api-iotfile-references-filtering.html) -k, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp list-files` command: ```text Examples: mdsp ls --assetid 1234...ef --passkey mypasskey list all files for assetid mdsp ls --assetid 1234...ef --filter "path=upload*" --passkey mypasskey list all files where path contains upload ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp markdown-help Command Syntax: ```bash mdsp markdown-help ``` Help: ```bash mdsp markdown-help --help ``` Alternative form: ```bash mc markdown-help ``` (The CLI was using `mc` as default command name in older versions) ## Description generates folder with markdown help commands for the CLI\* ## Usage Parameter list: ```text Usage: mdsp markdown-help|mdhelp [options] generates folder with markdown help commands for the CLI* Options: -d, --dir folder folder (default: "markdown-help") -o, --overwrite overwrite files if they already exist -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp markdown-help` command: ```text Examples: mdsp mdhelp ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp message-broker Command Syntax: ```bash mdsp message-broker ``` Help: ```bash mdsp message-broker --help ``` Alternative form: ```bash mc message-broker ``` (The CLI was using `mc` as default command name in older versions) ## Description manage message broker subscriptions and webhooks \* ## Usage Parameter list: ```text Usage: mdsp message-broker|mbk [options] manage message broker subscriptions and webhooks * Options: -m, --mode [info|modify|delete|template|send] mode [info | modify | delete | template | send] (default: "info") -s, --subscriptionid subscription id -e, --versionid version id -t, --topicid topic id (example: mdsp.core.am.v1.postbox.asset.deleted) -w, --webhook webhook url -k, --passkey passkey -f, --file .mdsp.json file with message definition (default: "messagebroker.message.mdsp.json") -o, --overwrite overwrite template file if it already exists -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp message-broker` command: ```text Examples: mdsp message-broker --mode info --subscriptionid --versionid --topicid get subscription webhook uri mdsp message-broker --mode delete --subscriptionid --versionid --topicid delete webhook mdsp message-broker --mode modify --subscriptionid --versionid --topicid --webhook configure webhook mdsp message-broker --mode template --file messagebroker.message.mdsp.json create template message file mdsp message-broker --mode send --file messagebroker.message.mdsp.json --topicid send message to the topic ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp mobile-app-instances Command Syntax: ```bash mdsp mobile-app-instances ``` Help: ```bash mdsp mobile-app-instances --help ``` Alternative form: ```bash mc mobile-app-instances ``` (The CLI was using `mc` as default command name in older versions) ## Description list, create or delete mobile app instances \* ## Usage Parameter list: ```text Usage: mdsp mobile-app-instances|mbi [options] list, create or delete mobile app instances * Options: -m, --mode [list|info|create|delete|update|template] mode [list | info | create | delete | update | template] (default: "list") -p, --appid mobile app id -i, --instanceid mobile app instance id -f, --file mobile app file [mobileapp.instance.mdsp.json] -o, --overwrite overwrite template files -k, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp mobile-app-instances` command: ```text Examples: mdsp mobile-app-instances --appid --mode list list mobile apps mdsp mobile-app-instances --appid --mode template create template file for mobile app instance mdsp mobile-app-instances --appid --mode create --file mobileapp.instance.mdsp.json create mobile app instance mdsp mobile-app-instances --appid --mode info --instanceid mobile app instance info mdsp mobile-app-instances --appid --mode delete --instanceid delete mobile app instance ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp mobile-apps Command Syntax: ```bash mdsp mobile-apps ``` Help: ```bash mdsp mobile-apps --help ``` Alternative form: ```bash mc mobile-apps ``` (The CLI was using `mc` as default command name in older versions) ## Description list, create or delete mobile apps \* ## Usage Parameter list: ```text Usage: mdsp mobile-apps|mb [options] list, create or delete mobile apps * Options: -m, --mode [list|info|create|delete|update|template] mode [list | info | create | delete | update | template] (default: "list") -t, --type [ios|android] type [ios|android] (default: "android") -p, --appid mobile app id -f, --file mobile app file [ios|android].mobileapp.mdsp.json -a, --all list full app information -o, --overwrite overwrite template files -k, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp mobile-apps` command: ```text Examples: mdsp mobile-apps --mode list list mobile apps mdsp mobile-apps --mode template --type [android|ios] create template file for mobileapp mdsp mobile-apps --mode create --file [android|ios].mobileapp.mdsp.json create mobileapp mdsp mobile-apps --mode info --appid mobile app info mdsp mobile-apps --mode delete --appid delete mobile app ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp models Command Syntax: ```bash mdsp models ``` Help: ```bash mdsp models --help ``` Alternative form: ```bash mc models ``` (The CLI was using `mc` as default command name in older versions) ## Description list, create or delete analytic models \* ## Usage Parameter list: ```text Usage: mdsp models|ml [options] list, create or delete analytic models * Options: -m, --mode [list|create|delete|update|info|download|template] mode [list | create | delete | update | info | download| template] (default: "list") -n, --modelname modelname -t, --modeltype modeltype -d, --modeldesc modeldesc (default: "created with Insights Hub CLI") -i, --modelid Insights Hub model id -f, --metadata model metadata file (default: "model.metadata.mdsp.json") -r, --version model version for download (default: "last") -p, --payload model payload file (default: "model.payload.mdsp.json") -i, --modelid Insights Hub model id -a, --modelauthor model author (default: "created by Insights Hub CLI") -k, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp models` command: ```text Examples: mdsp models --mode template --modeltype core.basicmodel --modelname MyModel creates a template for model mdsp models --mode create --metadata model.metadata.mdsp.json --payload model.payload.mdsp.json creates a model from specified files mdsp models --mode list lists all models in Insights Hub mdsp models --mode delete --modelid 1234567..ef deletes model with specified id mdsp models --mode info --modelid 123456...ef print out infos about model with id 132456...ef mdsp models --mode download --modelid 123456...ef download model with id 132456...ef ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp mqtt-createjwt Command Syntax: ```bash mdsp mqtt-createjwt ``` Help: ```bash mdsp mqtt-createjwt --help ``` Alternative form: ```bash mc mqtt-createjwt ``` (The CLI was using `mc` as default command name in older versions) ## Description creates a signed token for opcua pub sub authentication # ## Usage Parameter list: ```text Usage: mdsp mqtt-createjwt|jw [options] creates a signed token for opcua pub sub authentication # Options: -i, --clientid MQTT ClientId (default: "1cabd2d8-7b54-4ccd-b963-e3a0813375c1") -e, --expiration time until the token is valid in seconds (default: "3600") -c, --rootca path to CA root certificate (default: "CA-root.pem") -d, --devicecrt path to device certificate (default: "device.crt.pem") -k, --devicekey path to device certificate key (default: "device.key.pem") -p, --passphrase [passphrase] passphrase for device certificate key -n, --intermediate [intermediate] intermediate ca -t, --tenant [tenant] tenant name -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp mqtt-createjwt` command: ```text Examples: mdsp mqtt-createjwt --clientid "12345...ef" \ --rootca path/to/root.cer.pem \ --devicecrt path/to/device.cer.pem \ --devicekey path/to/devicekey.pem \ --passphrase "device key passphrase" \ --tenant yourtenant More Information: https://developer.mindsphere.io/howto/howto-connect-via-mqtt.html#security-concept-onboarding-certificatebearer ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp notifications Command Syntax: ```bash mdsp notifications ``` Help: ```bash mdsp notifications --help ``` Alternative form: ```bash mc notifications ``` (The CLI was using `mc` as default command name in older versions) ## Description send email, sms and push notifications \* ## Usage Parameter list: ```text Usage: mdsp notifications|nt [options] send email, sms and push notifications * Options: -m, --mode [template|status|send] mode [template|send] (default: "template") -t, --type [email|sms|push] type [email|sms|push] (default: "email") -i, --jobid job id for status command -f, --metadata metadata file with notification infos -a, --files comma separated list of attachement files for email (max 5) -o, --overwrite overwrite template files -k, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp notifications` command: ```text Examples: mdsp notifications --mode template --type [mail|sms|push] create template file with notification metadata mdsp notifications --mode send --metadata --type [mail|sms|push] send notifications (mail, sms, push) to recipients ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp oe-app-deploy Command Syntax: ```bash mdsp oe-app-deploy ``` Help: ```bash mdsp oe-app-deploy --help ``` Alternative form: ```bash mc oe-app-deploy ``` (The CLI was using `mc` as default command name in older versions) ## Description list, create, update app installation task(s) (open edge) \* ## Usage Parameter list: ```text Usage: mdsp oe-app-deploy|oead [options] list, create, update app installation task(s) (open edge) * Options: -m, --mode [list|create|update|accept|remove|template|info|check] list | create | update | accept | remove | template | info | check (default: "list") -i, --id the installation task id -d, --deviceid deviceid to filter -r, --releaseid software release id -f, --file .mdsp.json file with app data -s, --status [closed|open] closed | open -o, --overwrite overwrite template file if it already exists -k, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp oe-app-deploy` command: ```text Examples: mdsp oe-app-deploy --mode list --deviceid "7d018c..." list all installation and removal tasks of a specified device. mdsp oe-app-deploy --mode template create template files to define an app installation/removal task. mdsp oe-app-deploy --mode create --file edge.install.app.mdsp.json creates a new installation app taks. mdsp oe-app-deploy --mode remove --file edge.remove.app.mdsp.json creates a new removal app task. mdsp oe-app-deploy --mode update --id "7d018c..." --file edge.app.status.mdsp.json update an installation/removal task from status template file. mdsp oe-app-deploy --mode info --id get details of an installation task. mdsp oe-app-deploy --mode check --deviceid --releaseid check terms and condition of a software release on a a specific device. mdsp oe-app-deploy --mode accept --deviceid --releaseid accept terms and condition of a software release on a a specific device. ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp oe-app-inst Command Syntax: ```bash mdsp oe-app-inst ``` Help: ```bash mdsp oe-app-inst --help ``` Alternative form: ```bash mc oe-app-inst ``` (The CLI was using `mc` as default command name in older versions) ## Description list, create, configure or delete app instance (open edge) \* ## Usage Parameter list: ```text Usage: mdsp oe-app-inst|oeai [options] list, create, configure or delete app instance (open edge) * Options: -m, --mode [list|create|config|delete|template|info] list | create | config | delete | template | info (default: "list") -i, --id the app instance id -d, --deviceid the device id -f, --file .mdsp.json file with app inst data -o, --overwrite overwrite template file if it already exists -k, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp oe-app-inst` command: ```text Examples: mdsp oe-app-inst --mode list --deviceid list all app instances of device with deviceId. mdsp oe-app-inst --mode template create a template file for new app instance data. mdsp oe-app-inst --mode create --file edge.app.mdsp.json creates a new app instance from template file. mdsp oe-app-inst --mode config --id --file edge.appconfig.mdsp.json configure an app instance from template file. mdsp oe-app-inst --mode info --id get details of an app instance. mdsp oe-app-inst --mode delete --id delete app instance configuration. ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp oe-deploy-workflow Command Syntax: ```bash mdsp oe-deploy-workflow ``` Help: ```bash mdsp oe-deploy-workflow --help ``` Alternative form: ```bash mc oe-deploy-workflow ``` (The CLI was using `mc` as default command name in older versions) ## Description list, create/instantiate, update or delete/cancel workflow deployment model or instance(s) (open edge) \* ## Usage Parameter list: ```text Usage: mdsp oe-deploy-workflow|oedw [options] list, create/instantiate, update or delete/cancel workflow deployment model or instance(s) (open edge) * Options: -m, --mode [list|create|instantiate|update|cancel|delete|template|info] list | create | instantiate | update | cancel | delete | template | info (default: "list") -k, --key the workflow model key -i, --id the deployment model instance id -f, --file .mdsp.json file -md, --model include the model -u, --history include the transition history -c, --currentstate current state value to filter -g, --group group value to filter -d, --deviceid deviceid to filter -o, --overwrite overwrite template file if it already exists -k, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp oe-deploy-workflow` command: ```text Examples: mdsp oe-deploy-workflow --mode list list all workflow instances descriptions belonging to the caller's tenant. mdsp oe-deploy-workflow --mode template create a template files to define the workflow state machine model. mdsp oe-deploy-workflow --mode create --file edge.app.model.mdsp.json creates a new workflow model from template file. mdsp oe-deploy-workflow --mode instantiate --file edge.app.instance.mdsp.json creates a new workflow instance from template file. mdsp oe-deploy-workflow --mode update --id "7d018c..." --file edge.app.status.mdsp.json update a workflow instance. mdsp oe-deploy-workflow --mode info --id get details of a deployment workflow instance. mdsp oe-deploy-workflow --mode info --key get details of a deployment workflow model. mdsp oe-deploy-workflow --mode cancel --id cancel a deployment workflow. mdsp oe-deploy-workflow --mode delete --key delete a deployment workflow model. ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp oe-device-status Command Syntax: ```bash mdsp oe-device-status ``` Help: ```bash mdsp oe-device-status --help ``` Alternative form: ```bash mc oe-device-status ``` (The CLI was using `mc` as default command name in older versions) ## Description list, get, or update (open edge) device status information \* ## Usage Parameter list: ```text Usage: mdsp oe-device-status|oeds [options] list, get, or update (open edge) device status information * Options: -m, --mode [list|info|update|template] list | info | update | template (default: "list") -i, --deviceid the device id -t, --target [health|health-config-data|inventory|connection-status] type of status information to retrieve or to update. [ health | health-config-data | inventory | connection-status] (default: "health") -w, --softwaretype [APP|FIRMWARE] software type [ APP | FIRMWARE ] -s, --softwareid software id -f, --file openedge.*.mdsp.json file with update information definition -o, --overwrite overwrite template file if it already exists -k, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp oe-device-status` command: ```text Examples: mdsp oe-device-status --mode list --deviceid list all installed software on the device mdsp oe-device-status --mode list --deviceid 12345... --softwaretype APP list all apps installed on the device mdsp oe-device-status --mode info --target health --deviceid get the device health status mdsp oe-device-status --mode info --target health-config-data --deviceid get the device health config data mdsp oe-device-status --mode info --target inventory --deviceid get the software inventory of the device mdsp oe-device-status --mode info --target connection-status --deviceid get the device connection status mdsp oe-device-status --mode template --target inventory create template file for software inventory mdsp oe-device-status --mode template --target connection-status create template file for connection status mdsp oe-device-status --mode update --target inventory --file openedge.inventory.mdsp.json --deviceid update the software inventory of the device mdsp oe-device-status --mode update --target connection-status --deviceid send a heartbeat to the device ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp oe-device-types Command Syntax: ```bash mdsp oe-device-types ``` Help: ```bash mdsp oe-device-types --help ``` Alternative form: ```bash mc oe-device-types ``` (The CLI was using `mc` as default command name in older versions) ## Description list, create or delete device types (open edge) \* ## Usage Parameter list: ```text Usage: mdsp oe-device-types|oedt [options] list, create or delete device types (open edge) * Options: -m, --mode [list|create|delete|template|info] list | create | delete | template | info (default: "list") -f, --file .mdsp.json file with device type definition -t, --tenant tenant tenant of the device type definition -n, --devicetype the device type name -c, --code device type code -a, --assettype the device type associated asset type id -i, --id the device type id -o, --overwrite overwrite template file if it already exists -k, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp oe-device-types` command: ```text Examples: mdsp oe-device-types --mode list list all device types mdsp oe-device-types --mode list --tenant siemens list all device types which belongs to the tenant "siemens" mdsp oe-device-types --mode info --id get details of device type with the specified device id mdsp oe-device-types --mode template --devicetype board create a template file for specified device type mdsp oe-device-types --mode create --file board.devicetype.mdsp.json create device type board in Insights Hub mdsp oe-device-types --mode delete --id delete the device type with the device id ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp oe-devices Command Syntax: ```bash mdsp oe-devices ``` Help: ```bash mdsp oe-devices --help ``` Alternative form: ```bash mc oe-devices ``` (The CLI was using `mc` as default command name in older versions) ## Description list, create or delete (open edge) devices \* ## Usage Parameter list: ```text Usage: mdsp oe-devices|oed [options] list, create or delete (open edge) devices * Options: -m, --mode [list|create|delete|template|info] list | create | delete | template | info (default: "list") -f, --file .mdsp.json file with device definition -n, --devicename device name -a, --assetid the id of the asset linked to the device -t, --typeid the device type id -d, --desc description (default: "created with Insights Hub CLI") -s, --serialnumber the id of the asset linked to the device -i, --id the device id -o, --overwrite overwrite template file if it already exists -k, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp oe-devices` command: ```text Examples: mdsp oe-devices --mode list --assetid list all devices linked to the asset mdsp oe-devices --mode info --id get device details mdsp oe-devices --mode template create a template file for a new device mdsp oe-devices --mode delete --id delete the device with the specified id mdsp oe-devices --mode create --file openedge.device.mdsp.json create new device using the file openedge.device.mdsp.json ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp oe-firm-deploy Command Syntax: ```bash mdsp oe-firm-deploy ``` Help: ```bash mdsp oe-firm-deploy --help ``` Alternative form: ```bash mc oe-firm-deploy ``` (The CLI was using `mc` as default command name in older versions) ## Description list, create, update firmware deployment task(s) (open edge) \* ## Usage Parameter list: ```text Usage: mdsp oe-firm-deploy|oefd [options] list, create, update firmware deployment task(s) (open edge) * Options: -m, --mode [list|create|update|info|check|accept|template] list | create | update | info | check | accept |template (default: "list") -i, --id the installation task id -d, --deviceid deviceid to filter -r, --realeaseid firmware realease id -t, --type filter for tasks of a specific type (firmware, app, etc) -s, --status [closed|open] filter based on task progress, one of “closed” or “open” -f, --file .mdsp.json file with app data -o, --overwrite overwrite template file if it already exists -k, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp oe-firm-deploy` command: ```text Examples: mdsp oe-firm-deploy --mode list --deviceid "7d018c..." list all firmware deployment taks on a specified device. mdsp oe-firm-deploy --mode template create template files to define an firmware installation/update task. mdsp oe-firm-deploy --mode create --file edge.install.firmware.mdsp.json creates a new firmware installtion tak. mdsp oe-firm-deploy --mode update --id "7d018c..." --file edge.firmware.status.mdsp.json update a firmware installation task from status template file. mdsp oe-firm-deploy --mode info --id get details of a firmware installation task. mdsp oe-firm-deploy --mode check --deviceid --realeaseid check terms and condition of a firmware realease on a a specific device. mdsp oe-firm-deploy --mode accept --deviceid --realeaseid accept terms and condition of a firmware realease on a a specific device. ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp offboard-agent Command Syntax: ```bash mdsp offboard-agent ``` Help: ```bash mdsp offboard-agent --help ``` Alternative form: ```bash mc offboard-agent ``` (The CLI was using `mc` as default command name in older versions) ## Description offboards the agent in the Insights Hub \* ## Usage Parameter list: ```text Usage: mdsp offboard-agent|of [options] offboards the agent in the Insights Hub * Options: -c, --config config file for agent configuration -i, --agentid agent id -k, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp offboard-agent` command: ```text Examples: mdsp offboard-agent --config agent.json --passkey passkey... offboard agent with agent.json configuration mdsp offboard-agent --agentid 12345..ef --passkey passkey... offboard agent with 12345..ef agentid ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp onboard Command Syntax: ```bash mdsp onboard ``` Help: ```bash mdsp onboard --help ``` Alternative form: ```bash mc onboard ``` (The CLI was using `mc` as default command name in older versions) ## Description onboard the agent with configuration stored in the config file ## Usage Parameter list: ```text Usage: mdsp onboard|ob [options] onboard the agent with configuration stored in the config file Options: -c, --config config file with agent configuration (default: "agentconfig.json") -r, --cert [privatekey] required for agents with RSA_3072 profile. create with: openssl genrsa -out private.key 3072 -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp onboard` command: ```text Examples: mdsp ob uses default agentconfig.json mdsp onboard --config agent.json uses specified configuration file mdsp onboard --config agent.json --cert private.key uses specified key for RSA_3072 profile ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp policy Command Syntax: ```bash mdsp policy ``` Help: ```bash mdsp policy --help ``` Alternative form: ```bash mc policy ``` (The CLI was using `mc` as default command name in older versions) ## Description list, create or delete policies \* ## Usage Parameter list: ```text Usage: mdsp policy|po [options] list, create or delete policies * Options: -m, --mode [list|create|update|delete|template|info] list | create | update | delete | template | info (default: "list") -f, --file .mdsp.json file with policy definition (default: "policy.mdsp.json") -n, --policy the policy name -i, --policyid the policy id -o, --overwrite overwrite template file if it already exists -k, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp policy` command: ```text Examples: mdsp policy --mode list list all policies mdsp policy --mode template --policy create a template file for mdsp policy --mode create --file create policy mdsp policy --mode update --file --policyid update policy mdsp policy --mode info --policyid policy info for specified id mdsp policy --mode delete --policyid delete policy with specified id ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp prepare-bulk Command Syntax: ```bash mdsp prepare-bulk ``` Help: ```bash mdsp prepare-bulk --help ``` Alternative form: ```bash mc prepare-bulk ``` (The CLI was using `mc` as default command name in older versions) ## Description creates a template directory for timeseries (bulk) upload \* ## Usage Parameter list: ```text Usage: mdsp prepare-bulk|pb [options] creates a template directory for timeseries (bulk) upload * Options: -d, --dir config file with agent configuration (default: "bulkupload") -w, --twintype twintype of asset [performance|simulation] -i, --assetid Insights Hub asset id -t, --typeid typeid e.g. castidev.Engine -s, --size entries per file (default: "100") -f, --files generated files (default: "2") -o, --offset offset in days from current date (default: "0") -y, --retry retry attempts before giving up (default: "3") -k, --passkey passkey -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp prepare-bulk` command: ```text Examples: mdsp prepare-bulk --typeid castidev.Engine this creates a directory called bulkimport for new asset of type castidev.Engine mdsp pb --dir asset1 -i 123456...abc this creates a directory called asset1 for existing asset mdsp pb -of 3 -t castidev.Engine start data creation template 3 days before now use --mode performance for standard data generation or --mode simulation for high frequency data generation The typeid must be derived from core.basicdevice and asset twintype must be simulation for high frequency data upload ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp register-diagnostic Command Syntax: ```bash mdsp register-diagnostic ``` Help: ```bash mdsp register-diagnostic --help ``` Alternative form: ```bash mc register-diagnostic ``` (The CLI was using `mc` as default command name in older versions) ## Description register agent for diagnostic \* ## Usage Parameter list: ```text Usage: mdsp register-diagnostic|rd [options] register agent for diagnostic * Options: -c, --config config file with agent configuration (default: "agentconfig.json") -i, --agentid agent id -k, --passkey passkey -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp register-diagnostic` command: ```text Examples: mdsp rd -k mypasskey mdsp register-diagnostic --config someagent.json -passkey mypasskey ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp renew-agent Command Syntax: ```bash mdsp renew-agent ``` Help: ```bash mdsp renew-agent --help ``` Alternative form: ```bash mc renew-agent ``` (The CLI was using `mc` as default command name in older versions) ## Description renews the agent secrets \* ## Usage Parameter list: ```text Usage: mdsp renew-agent|rn [options] renews the agent secrets * Options: -c, --config config file for agent configuration -k, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp renew-agent` command: ```text Examples: mdsp renew-agent --config agent.json --passkey passkey... renew agent secrets in agent.json configuration ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp run-bulk Command Syntax: ```bash mdsp run-bulk ``` Help: ```bash mdsp run-bulk --help ``` Alternative form: ```bash mc run-bulk ``` (The CLI was using `mc` as default command name in older versions) ## Description runs the timeseries (bulk) upload job from directory \* ## Usage Parameter list: ```text Usage: mdsp run-bulk|rb [options] runs the timeseries (bulk) upload job from directory * Options: -d, --dir config file with agent configuration (default: "bulkupload") -y, --retry retry attempts before giving up (default: "3") -l, --parallel parallel chunk uploads (default: "3") -s, --size entries per file (default: "9007199254740991") -f, --force force generation of json files, file upload and creation of jobs -k, --passkey passkey -i, --timeseries use (deprecated) timeseries upload -v, --verbose verbose output -t, --start start sending data to Insights Hub -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp run-bulk` command: ```text Examples: mdsp run-bulk runs the upload job from the bulkimport directory mdsp run-bulk --dir asset1 --verbose runs the upload job from the asset1 with verbose output ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp schedules Command Syntax: ```bash mdsp schedules ``` Help: ```bash mdsp schedules --help ``` Alternative form: ```bash mc schedules ``` (The CLI was using `mc` as default command name in older versions) ## Description list, create, start, stop or delete job schedules \* ## Usage Parameter list: ```text Usage: mdsp schedules|js [options] list, create, start, stop or delete job schedules * Options: -m, --mode [list|create|start|stop|template|info|delete] list | create | stop | template | info (default: "list") -f, --file .mdsp.json file with job schedule definition (default: "jobmanager.schedule.mdsp.json") -i, --scheduleid the schedule id -n, --name the name filter (contains) for list command -s, --status the status filter (equals, e.g. STOPPED, FAILED...) for list command -d, --modelid the modelid filter (equals) for list command -o, --overwrite overwrite template file if it already exists -k, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp schedules` command: ```text Examples: mdsp schedules --mode list list all job schedules mdsp schedules --mode list --status RUNNING list all RUNNING job schedules mdsp schedules --mode list --modelid list all job schedules for specified model mdsp schedules --mode template create template file for job schedule creation mdsp schedules --mode create --file create job schedule mdsp schedules --mode info --scheduleid get infos about the job schedule mdsp schedules --mode start --scheduleid start job schedule with job id mdsp schedules --mode stop --scheduleid stop job schedule with job id ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp sdi-data-lakes Command Syntax: ```bash mdsp sdi-data-lakes ``` Help: ```bash mdsp sdi-data-lakes --help ``` Alternative form: ```bash mc sdi-data-lakes ``` (The CLI was using `mc` as default command name in older versions) ## Description manage data lakes for SDI \* ## Usage Parameter list: ```text Usage: mdsp sdi-data-lakes|sdl [options] manage data lakes for SDI * Options: -m, --mode [list|create|update|template|info|delete] list | create | update | template | info | delete (default: "list") -d, --datalake data lake file with definition for --mode create or update command -i, --datalakeid the datalake id for --mode info, update or delete command -o, --overwrite overwrite template file if it already exists -k, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp sdi-data-lakes` command: ```text Examples: mdsp sdi-data-lakes --mode list list all sdi datalakes mdsp sdi-data-lakes --mode template create template file mdsp sdi-data-lakes --mode create --datalake create sdi data lake mdsp sdi-data-lakes --mode update --datalake --datalakeid update sdi data lake mdsp sdi-data-lakes --mode info --datalakeid get sdi data lake info mdsp sdi-data-lakes --mode delete --datalakeid delete sdi data lake ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp sdi-data-queries Command Syntax: ```bash mdsp sdi-data-queries ``` Help: ```bash mdsp sdi-data-queries --help ``` Alternative form: ```bash mc sdi-data-queries ``` (The CLI was using `mc` as default command name in older versions) ## Description manage data queries for SDI \* ## Usage Parameter list: ```text Usage: mdsp sdi-data-queries|sdq [options] manage data queries for SDI * Options: -m, --mode [list|create|update|template|info|delete|latest] list | create | update | template | info | delete | latest (default: "list") -d, --query data query file with definition for --mode create or update command -i, --queryid the query id for --mode info, update or delete command -o, --overwrite overwrite template file if it already exists -r, --result result file for --mode latest (default: "sdi.query.latest.mdsp.json") -k, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp sdi-data-queries` command: ```text Examples: mdsp sdi-data-queries --mode list list all sdi dataqueries mdsp sdi-data-queries --mode template create template file mdsp sdi-data-queries --mode create --query create sdi data query mdsp sdi-data-queries --mode update --query --queryid update sdi data query mdsp sdi-data-queries --mode info --queryid get sdi data query info mdsp sdi-data-queries --mode latest --queryid get latest query results mdsp sdi-data-queries --mode delete --queryid delete sdi data query ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp sdi-data-registries Command Syntax: ```bash mdsp sdi-data-registries ``` Help: ```bash mdsp sdi-data-registries --help ``` Alternative form: ```bash mc sdi-data-registries ``` (The CLI was using `mc` as default command name in older versions) ## Description manage data registries for SDI \* ## Usage Parameter list: ```text Usage: mdsp sdi-data-registries|sdr [options] manage data registries for SDI * Options: -m, --mode [list|create|update|template|info|delete] list | create | update | template | info | delete (default: "list") -d, --dataregistry data registry file with definition for --mode create or update command -i, --registryid the dataregistry id for --mode info, update or delete command -o, --overwrite overwrite template file if it already exists -k, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp sdi-data-registries` command: ```text Examples: mdsp sdi-data-registries --mode list list all sdi dataregistries mdsp sdi-data-registries --mode template create template file mdsp sdi-data-registries --mode create --dataregistry create sdi data registry mdsp sdi-data-registries --mode update --dataregistry --registryid update sdi data registry mdsp sdi-data-registries --mode info --registryid get sdi data registry info mdsp sdi-data-registries --mode delete --registryid delete sdi data registry ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp sdi-data-types Command Syntax: ```bash mdsp sdi-data-types ``` Help: ```bash mdsp sdi-data-types --help ``` Alternative form: ```bash mc sdi-data-types ``` (The CLI was using `mc` as default command name in older versions) ## Description manage data types for SDI \* ## Usage Parameter list: ```text Usage: mdsp sdi-data-types|sdy [options] manage data types for SDI * Options: -m, --mode [list|create|template|info|delete|addpatterns|suggest] command mode (default: "list") -f, --for [datatype|pattern|suggest] parameter for --mode template (default: "datatype") -d, --datatype data type file with definition for --mode create or update command -p, --pattern pattern file with definition for --mode addpatterns command -s, --samplevalues file with sample values -t, --testvalues file with sample values -i, --datatypename the datatype id -o, --overwrite overwrite template file if it already exists -k, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp sdi-data-types` command: ```text Examples: mdsp sdi-data-types --mode list list all sdi data types mdsp sdi-data-types --mode template --for datatype create template file for --mode create command mdsp sdi-data-types --mode template --for pattern create pattern file for --mode addpattern command mdsp sdi-data-types --mode template --for suggest create samples files for --mode suggest command mdsp sdi-data-types --mode create --datatype create sdi data type mdsp sdi-data-types --mode info --datatypename get sdi data type info mdsp sdi-data-types --mode delete --datatypename delete sdi data type mdsp sdi-data-types --mode suggest --samplevalues --testvalues suggest patterns from sample data ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp sdi-execution-jobs Command Syntax: ```bash mdsp sdi-execution-jobs ``` Help: ```bash mdsp sdi-execution-jobs --help ``` Alternative form: ```bash mc sdi-execution-jobs ``` (The CLI was using `mc` as default command name in older versions) ## Description manage data execution jobs for SDI \* ## Usage Parameter list: ```text Usage: mdsp sdi-execution-jobs|sdx [options] manage data execution jobs for SDI * Options: -m, --mode [list|create|template|info|delete|result] list | create | template | info | delete | result (default: "list") -d, --executionjob data execution job file with definition for --mode create command -q, --queryid the query id -i, --jobid the executionjob id for --mode info, update or delete command -r, --result result file for --mode result (default: "sdi.jobresult.mdsp.json") -o, --overwrite overwrite template file if it already exists -k, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp sdi-execution-jobs` command: ```text Examples: mdsp sdi-execution-jobs --mode list list all sdi data execution jobs mdsp sdi-execution-jobs --mode template create template file mdsp sdi-execution-jobs --mode create --executionjob create sdi data execution job mdsp sdi-execution-jobs --mode update --executionjob --jobid update sdi data executionjob mdsp sdi-execution-jobs --mode info --jobid get sdi data execution job info mdsp sdi-execution-jobs --mode result --jobid get execution job results mdsp sdi-execution-jobs --mode delete --jobid delete sdi data execution job ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp sdi-file-upload Command Syntax: ```bash mdsp sdi-file-upload ``` Help: ```bash mdsp sdi-file-upload --help ``` Alternative form: ```bash mc sdi-file-upload ``` (The CLI was using `mc` as default command name in older versions) ## Description upload file to SDI \* ## Usage Parameter list: ```text Usage: mdsp sdi-file-upload|sdu [options] upload file to SDI * Options: -f, --file file to upload to SDI -h, --filepath file path in SDI -m, --mime [mime-type] mime type of the file (default: automatic recognition) -y, --retry retry attempts before giving up (default: "3") -k, --passkey passkey -v, --verbose verbose output --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp sdi-file-upload` command: ```text Examples: mdsp sdi-file-upload --file upload file to SDI Restriction: This works only for SDI only tenants (without Integrated Data Lake) If you are Insights Hub Integrated Data Lake Customer use mdsp data-lake --mode upload command instead. ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp sdi-ingest-jobs Command Syntax: ```bash mdsp sdi-ingest-jobs ``` Help: ```bash mdsp sdi-ingest-jobs --help ``` Alternative form: ```bash mc sdi-ingest-jobs ``` (The CLI was using `mc` as default command name in older versions) ## Description manage ingest jobs for SDI \* ## Usage Parameter list: ```text Usage: mdsp sdi-ingest-jobs|sdj [options] manage ingest jobs for SDI * Options: -m, --mode [list|create|template|info] list | create | template | info (default: "list") -j, --ingestjob the job information for --mode create command -i, --jobid the job id for --mode info command -o, --overwrite overwrite template file if it already exists -k, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp sdi-ingest-jobs` command: ```text Examples: mdsp sdi-ingest-jobs --mode list list all sdi data ingest jobs mdsp sdi-ingest-jobs --mode template create template file mdsp sdi-ingest-jobs --mode create --ingestjob create sdi ingest job mdsp sdi-ingest-jobs --mode info --jobid get sdi ingest job info ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp sdi-iot-registries Command Syntax: ```bash mdsp sdi-iot-registries ``` Help: ```bash mdsp sdi-iot-registries --help ``` Alternative form: ```bash mc sdi-iot-registries ``` (The CLI was using `mc` as default command name in older versions) ## Description manage iot data registries for SDI \* ## Usage Parameter list: ```text Usage: mdsp sdi-iot-registries|sdt [options] manage iot data registries for SDI * Options: -m, --mode [list|create|template|info|delete] list | create | template | info | delete (default: "list") -d, --iotregistry data registry file with definition for --mode create or update command -i, --registryid the iotregistry id for --mode info, update or delete command -o, --overwrite overwrite template file if it already exists -k, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp sdi-iot-registries` command: ```text Examples: mdsp sdi-iot-registries --mode list list all sdi dataregistries mdsp sdi-iot-registries --mode template create template file mdsp sdi-iot-registries --mode create --iotregistry create sdi iot registry mdsp sdi-iot-registries --mode update --iotregistry --registryid update sdi iot registry mdsp sdi-iot-registries --mode info --registryid get sdi iot registry info mdsp sdi-iot-registries --mode delete --registryid delete sdi iot registry ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp sdi-ontologies Command Syntax: ```bash mdsp sdi-ontologies ``` Help: ```bash mdsp sdi-ontologies --help ``` Alternative form: ```bash mc sdi-ontologies ``` (The CLI was using `mc` as default command name in older versions) ## Description manage ontologies for SDI \* ## Usage Parameter list: ```text Usage: mdsp sdi-ontologies|sdo [options] manage ontologies for SDI * Options: -m, --mode [list|infer|update|template|info|delete] list | infer | update | template | info | delete (default: "list") -d, --ontology ontology file with definition for --mode infer or update command -i, --ontologyid the ontology id for --mode info, update or delete command -o, --overwrite overwrite template file if it already exists -k, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp sdi-ontologies` command: ```text Examples: mdsp sdi-ontologies --mode list list all sdi ontologies mdsp sdi-ontologies --mode template create template file mdsp sdi-ontologies --mode infer --ontology infer sdi ontology mdsp sdi-ontologies --mode info --ontologyid get sdi ontology info mdsp sdi-ontologies --mode delete --ontologyid delete sdi ontology ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp sdi-ontology-jobs Command Syntax: ```bash mdsp sdi-ontology-jobs ``` Help: ```bash mdsp sdi-ontology-jobs --help ``` Alternative form: ```bash mc sdi-ontology-jobs ``` (The CLI was using `mc` as default command name in older versions) ## Description manage ontology jobs for SDI \* ## Usage Parameter list: ```text Usage: mdsp sdi-ontology-jobs|sdb [options] manage ontology jobs for SDI * Options: -m, --mode [submit|info] submit|info (default: "submit") -o, --ontology ontology file (json or owl) -n, --name the ontology name (default: "myontology") -i, --ontologyid the ontology id (for update) -k, --keymappingtype key mapping type (default: "INNER JOIN") -s, --description the ontology description (default: "created-by-mindsphere-cli") -j, --jobid the jobid for --info command -k, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp sdi-ontology-jobs` command: ```text Examples: mdsp sdi-ontology-jobs --mode submit --ontology upload ontology mdsp sdi-ontology-jobs --mode info --jobid get sdi ontology upload job info ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp sdi-search-schemas Command Syntax: ```bash mdsp sdi-search-schemas ``` Help: ```bash mdsp sdi-search-schemas --help ``` Alternative form: ```bash mc sdi-search-schemas ``` (The CLI was using `mc` as default command name in older versions) ## Description search SDI schemas \* ## Usage Parameter list: ```text Usage: mdsp sdi-search-schemas|sds [options] search SDI schemas * Options: -m, --mode [template|search] search | template (default: "search") -s, --searchrequest search request for --search command (default: "sdi.searchrequest.mdsp.json") -o, --overwrite overwrite template file if it already exists -k, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp sdi-search-schemas` command: ```text Examples: mdsp sdi-search-schemas --mode template create template file mdsp sdi-search-schemas --mode search --searchrequest search for sdi schemas ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp service-credentials Command Syntax: ```bash mdsp service-credentials ``` Help: ```bash mdsp service-credentials --help ``` Alternative form: ```bash mc service-credentials ``` (The CLI was using `mc` as default command name in older versions) ## Description provide login for commands which require technical user credentials \* ## Usage Parameter list: ```text Usage: mdsp service-credentials|sc [options] provide login for commands which require technical user credentials * Options: -m, --mode [config|list|select|add] config | list | select | add (default: "config") -i, --index select credentials with specified index -y, --type type (APP | SERVICE) (default: "APP") -o, --port port for config web server (default: "4994") -u, --user credentials: username -p, --password credendials: password -g, --gateway region string or full gateway url (e.g. eu1, eu2 or https://gateway.eu1.mindsphere.io) -t, --tenant your tenant name -s, --usertenant your user tenant name -a, --appName your application name (e.g. cli) -p, --appVersion your application version (e.g. 1.0.0) -k, --passkey passkey (you will use this in the commands which require service credentials) -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp service-credentials` command: ```text Example: mdsp service-credentials --mode config start configuration web server on http://localhost:4994 mdsp service-credentials --mode config --port 10000 start configuration web server on http://localhost:10000 mdsp service-credentials --mode list list all configured credentials mdsp service-credentials --mode select --index select credentials with index from the list mdsp service-credentials --mode add --type APP ... add new APP credentials ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp service-token Command Syntax: ```bash mdsp service-token ``` Help: ```bash mdsp service-token --help ``` Alternative form: ```bash mc service-token ``` (The CLI was using `mc` as default command name in older versions) ## Description displays the service token for use in other tools (e.g. postman) \* ## Usage Parameter list: ```text Usage: mdsp service-token|stk [options] displays the service token for use in other tools (e.g. postman) * Options: -k, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp service-token` command: ```text Examples: mdsp service-token --passkey mypasskey displays the service token (encoded only) mdsp service-token --passkey mypasskey --verbose displays the service token (encoded and decoded) ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp signal-calculation Command Syntax: ```bash mdsp signal-calculation ``` Help: ```bash mdsp signal-calculation --help ``` Alternative form: ```bash mc signal-calculation ``` (The CLI was using `mc` as default command name in older versions) ## Description process timeseries data \* ## Usage Parameter list: ```text Usage: mdsp signal-calculation|cal [options] process timeseries data * Options: -m, --mode [template|calculate] template | calculate (default: "template") -o, --on [data|asset] on [data | asset] (default: "data") -f, --template .mdsp.json template file default: (signal.[template|templatedirect].mdsp.json) -u, --result .mdsp.json with signal calculation result (default: "signal.result.mdsp.json") -i, --assetid assetid for template creation -a, --aspect aspect for template creation -r, --variable variables for template creation -s, --size timeseries length for generated data (default: "5") -t, --timeseries comma separated list of time series files (if data is not embedded) -o, --overwrite overwrite template file if it already exists -k, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp signal-calculation` command: ```text Examples: mdsp signal-calculation --mode template create template file for signal calculation mdsp signal-calculation --mode template --on asset --assetid --aspect --variable variable creates template for calculation using Insights Hub timeseries data mdsp signal-calculation --mode calculate --template calculates new signal from the timeseries specified in template file mdsp signal-calculation --mode calculate \ --template \ --timeseries \ --assetid \ --aspect calculates new signal from the timeseries specified in external file mdsp signal-calculation --mode calculate --on asset --template calculates new signal from the Insights Hub timeseries Operation List: https://developer.mindsphere.io/apis/analytics-signalcalculation/api-signalcalculation-overview.html ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp starter-js Command Syntax: ```bash mdsp starter-js ``` Help: ```bash mdsp starter-js --help ``` Alternative form: ```bash mc starter-js ``` (The CLI was using `mc` as default command name in older versions) ## Description creates a starter project in javascript # ## Usage Parameter list: ```text Usage: mdsp starter-js|sj [options] creates a starter project in javascript # Options: -d, --dir directory name (default: "starterjs") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp starter-js` command: ```text Examples: mdsp starter-js this creates a directory called starterts mdsp starter-js --dir mindconnect-agent this creates a directory called mindconnect-agent ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp starter-ts Command Syntax: ```bash mdsp starter-ts ``` Help: ```bash mdsp starter-ts --help ``` Alternative form: ```bash mc starter-ts ``` (The CLI was using `mc` as default command name in older versions) ## Description creates a starter project in typescript # ## Usage Parameter list: ```text Usage: mdsp starter-ts|st [options] creates a starter project in typescript # Options: -d, --dir directory name (default: "starterts") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp starter-ts` command: ```text Examples: mdsp starter-ts this creates a directory called starterts mdsp starter-ts --dir mindconnect-agent this creates a directory called mindconnect-agent ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp subtenants Command Syntax: ```bash mdsp subtenants ``` Help: ```bash mdsp subtenants --help ``` Alternative form: ```bash mc subtenants ``` (The CLI was using `mc` as default command name in older versions) ## Description list, create or delete subtenants \* ## Usage Parameter list: ```text Usage: mdsp subtenants|st [options] list, create or delete subtenants * Options: -m, --mode [list|create|delete|template | info] list | create | delete | template | info (default: "list") -f, --file .mdsp.json file with subtenant definition -n, --subtenant the subtenant name -i, --subtenantid the subtenant id -k, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp subtenants` command: ```text Examples: mdsp subtenants --mode list list all subtenants mdsp subtenants --mode template --subtenant create a template file for mdsp subtenants --mode create --file create subtenant mdsp subtenants --mode info --subtenantid subtenant info for specified id mdsp subtenants --mode delete --subtenantid delete subtenant with specified id ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp tenant Command Syntax: ```bash mdsp tenant ``` Help: ```bash mdsp tenant --help ``` Alternative form: ```bash mc tenant ``` (The CLI was using `mc` as default command name in older versions) ## Description create or delete tenant legal configuration and logo \* ## Usage Parameter list: ```text Usage: mdsp tenant|ti [options] create or delete tenant legal configuration and logo * Options: -m, --mode [info|create|delete|upload|template] mode [info | create | delete | upload | template] (default: "info") -f, --file .mdsp.json file with legal configuration or logo -k, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp tenant` command: ```text Examples: mdsp tenant prints out the tenant information mdsp tenant --mode info prints out the tenant information mdsp tenant --mode template creates template file with legal configuration mdsp tenant --mode delete deletes legal configuration mdsp tenant --mode create --file creates legal configuration mdsp tenant --mode upload --file uploads the company logo ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp timeseries Command Syntax: ```bash mdsp timeseries ``` Help: ```bash mdsp timeseries --help ``` Alternative form: ```bash mc timeseries ``` (The CLI was using `mc` as default command name in older versions) ## Description list timeseries data \* ## Usage Parameter list: ```text Usage: mdsp timeseries|ts [options] list timeseries data * Options: -i, --assetid Insights Hub asset id -n, --aspectname Insights Hub aspect name -f, --from begining of the time range to read (default: "2022-11-05T01:51:16.182Z") -t, --to end of the time range to read (default: "2022-11-06T01:51:16.182Z") -s, --select comma separated list of variable names -d, --download download timeseries to a set of files -a, --all include also quality codes not just variable values -r, --raw don't strip the nextRecord URLs from downloaded JSON -h, --formatted write JSON strings with indentation -l, --local use local time in timeseries list -c, --count number of timeseries entries per request (default: "2000") -p, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp timeseries` command: ```text Examples: mdsp timeseries --asssetid 1234567..ef --aspectname Environment list recent timeseries for aspect Environment mdsp timeseries --asssetid 1234567..ef --aspectname Environment --select Temperature list recent temperature timeseries mdsp timeseries --asssetid 1234567..ef --aspectname Environment --select Temperature --all list all recent temperature timeseries mdsp timeseries --asssetid 1234567..ef --aspectname Environment --select Temperature --all list all recent temperature timeseries mdsp timeseries --asssetid 1234567..ef --aspectname Environment --download download the recent timeseries data for the Environment aspect Important: Please use bulk commands if you want to download a lot of data. ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp unregister-diagnostic Command Syntax: ```bash mdsp unregister-diagnostic ``` Help: ```bash mdsp unregister-diagnostic --help ``` Alternative form: ```bash mc unregister-diagnostic ``` (The CLI was using `mc` as default command name in older versions) ## Description unregister agent from diagnostic \* ## Usage Parameter list: ```text Usage: mdsp unregister-diagnostic|ud [options] unregister agent from diagnostic * Options: -c, --config config file with agent configuration (default: "agentconfig.json") -i, --agentid agent id -k, --passkey passkey -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp unregister-diagnostic` command: ```text Examples: mdsp ud -k mypasskey mdsp unregister-diagnostic --config someagent.json -passkey mypasskey ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp upload-file Command Syntax: ```bash mdsp upload-file ``` Help: ```bash mdsp upload-file --help ``` Alternative form: ```bash mc upload-file ``` (The CLI was using `mc` as default command name in older versions) ## Description upload the file to the Insights Hub file service (optional: passkey) \* ## Usage Parameter list: ```text Usage: mdsp upload-file|uf [options] upload the file to the Insights Hub file service (optional: passkey) * Options: -c, --config config file with agent configuration -r, --cert [privatekey] required for agents with RSA_3072 profile. create with: openssl genrsa -out private.key 3072 -f, --file file to upload to the file service -h, --filepath file path in the Insights Hub -l, --parallel parallel chunk uploads (default: "3") -i, --assetid [assetid] Insights Hub asset id (default: upload to the agent) -m, --mime [mime-type] mime type of the file (default: automatic recognition) -d, --desc [description] description -k, --chunked Use chunked upload -y, --retry retry attempts before giving up (default: "3") -p, --passkey passkey (optional, file upload uses service credentials *) -v, --verbose verbose output --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp upload-file` command: ```text Examples: mdsp uf -f CHANGELOG.md upload file CHANGELOG.md to the agent mdsp upload-file --file CHANGELOG.md --assetid 5...f --mime text/plain upload file to a specified asset with custom mime type mdsp upload-file --file CHANGELOG.md --chunked upload file using experimental chunked upload ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp upload-timeseries Command Syntax: ```bash mdsp upload-timeseries ``` Help: ```bash mdsp upload-timeseries --help ``` Alternative form: ```bash mc upload-timeseries ``` (The CLI was using `mc` as default command name in older versions) ## Description parse .csv file with timeseriesdata and upload the timeseries data to Insights Hub ## Usage Parameter list: ```text Usage: mdsp upload-timeseries|uts [options] parse .csv file with timeseriesdata and upload the timeseries data to Insights Hub Options: -c, --config config file with agent configuration (default: "agentconfig.json") -r, --cert [privatekey] required for agents with RSA_3072 profile. create with: openssl genrsa -out private.key 3072 -f, --file csv file containing the timeseries data to upload to Insights Hub -s, --size max records per http post (default: "200") -n, --no-validation switch validation off (only if you are sure that the timeseries upload works) -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp upload-timeseries` command: ```text Examples: mdsp ts -f timeseries.csv upload timeseries from the csv file to Insights Hub mdsp upload-timeseries --file timeseries.csv --size 100 use http post size of 100 records Data Format: (use your own data point ids from Insights Hub) timestamp, dataPointId, qualityCode, value 2022-11-06T01:51:14.192Z, DP-Temperature ,0, 20.34 2022-11-06T01:51:15.192Z, DP-Humidity, 0, 70 2022-11-06T01:51:16.192Z, DP-Pressure, 0, 1012.3 Make sure that the timestamp is in ISO format. The headers and the casing (timestamp, dataPointId) are important. The values must correspond with data types configured in Insights Hub (in example: DP-Humidity must be an integer) Important: You have to configure the data source and data mappings in Insights Hub asset manager before you can upload the data See also: https://documentation.mindsphere.io/resources/html/asset-manager/en-US/116404525451.html ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # mdsp visual-flow-creator Command Syntax: ```bash mdsp visual-flow-creator ``` Help: ```bash mdsp visual-flow-creator --help ``` Alternative form: ```bash mc visual-flow-creator ``` (The CLI was using `mc` as default command name in older versions) ## Description manage vfc projects and nodes \* ## Usage Parameter list: ```text Usage: mdsp visual-flow-creator|vfc [options] manage vfc projects and nodes * Options: -m, --mode [list|create|update|delete|template|get-nodes|put-nodes|info] list | create | update | delete | template | get-nodes | put-nodes | info (default: "list") -u, --user user email -f, --file .mdsp.json file with vfc project definition (default: "vfc.project.mdsp.json") -n, --nodes .mdsp.json file with vfc project nodes (default: "vfc.nodes.mdsp.json") -p, --project the project name (default: "vfc-project-1667699476187") -i, --id the vfc project id -o, --overwrite overwrite template file if it already exists -k, --passkey passkey -y, --retry retry attempts before giving up (default: "3") -v, --verbose verbose output -h, --help display help for command ``` ## Examples Here are some examples of how to use the `mdsp visual-flow-creator` command: ```text Examples: mdsp vfc --user --mode list list all vfc projects mdsp vfc --user --mode template --project create a template file for vfc project mdsp vfc --user --mode create --file create vfc project mdsp vfc --user --mode update --file --id update project mdsp vfc --user --mode info --id vfc project info for specified id mdsp vfc --user --mode delete --handle delete vfc project with specified id ``` See [Insights Hub API documentation](https://documentation.mindsphere.io/MindSphere/apis/index.html) for more information about Insights Hub APIs. # TypeScript SDK - Overview The library comes with the typescript SDK which can be used to access Insights Hub APIs. It implements support for both frontend (browser e.g. angular, react...) and backend development in node.js while supporting different Insights Hub authentication types. **Frontend:** - Browser (Session, Cookies) **Backend (node.js):** - UserCredentials - AppCredentials - ServiceCredentials - Insights Hub Agents The SDK implements the Clients for following APIs: ## Platform Core APIs | Name | SDK - Client | Command | | --------------------------- | ------------ | ------- | | Identity Management | ✅ | ✅ | | Resource Access Management1 | ✅ | ✅ | | Oauth Authorization | ✅ | ✅ | | Tenant Management | ✅ | ✅ | | Token Management | ✅ | ✅ | | Message Broker2 (preview) | ✅ | ✅ | | Usage Transparency | ✅ | | 1 In the first stage of the availability Resource Access Management must be enabled for the tenant via Insights Hub support team. 2 Message Broker is only available on preview tenants ## IoT and Storage | Name | SDK - Client | Command | | -------------------------- | ------------ | ------- | | IoT File | ✅ | ✅ | | IoT Time Series | ✅ | ✅ | | IoT TS Aggregates (v3, v4) | ✅ | ✅ | | IoT TS Bulk | ✅ | ✅ | | Integrated Data Lake | ✅ | ✅ | ## Connectivity | Name | SDK - Client | Command | | --------------------- | ------------ | ------- | | Agent Management | ✅ | ✅ | | MindConnect API | ✅ | ✅ | | Commanding API (sync) | ✅ | ✅ | | OPC UA PubSub | | ✅ | ## Advanced Services | Name | SDK - Client | Command | | -------------------- | ------------ | ------- | | Asset Management | ✅ | ✅ | | Event Management | ✅ | ✅ | | Notification | ✅ | ✅ | | WorkOrder Management | ✅ | ✅ | | Visual Flow Creator | ✅ | ✅ | ## Analytics Services | Name | SDK - Client | Command | | ------------------ | ------------ | ------- | | Anomaly Detection | ✅ | ✅ | | Data Exchange | ✅ | ✅ | | Event Analytics | ✅ | ✅ | | Job Manager | ✅ | ✅ | | KPI Calculation | ✅ | ✅ | | Model Management | ✅ | ✅ | | Signal Calculation | ✅ | ✅ | | Signal Validation | ✅ | ✅ | | Spectrum Analysis | ✅ | ✅ | | Trend Prediction | ✅ | ✅ | ## MindConnect Open Edge | Name | SDK - Client | Command | | ---------------------------- | ------------ | ------- | | Device Management | ✅ | ✅ | | Device Status | ✅ | ✅ | | Deployment Workflow | ✅ | ✅ | | Device Configuration | ✅ | ✅ | | Edge App Deployment | ✅ | ✅ | | Edge App Instance Management | ✅ | ✅ | | Firmware Deployment | ✅ | ✅ | ## Semantic Data Interconnect | Name | SDK - Client | Command | | ---------------------- | ------------ | ------- | | SDI Data Management | ✅ | ✅ | | SDI Data Query | ✅ | ✅ | | SDI Semantic Modelling | ✅ | ✅ | The example below shows how to use the SDK from browser: ```javascript // The example shows how to Get Assets from Insights Hub with custom AssetType // using frontend authentication // you can pass an instance an Authorizer // (BrowserAuth, UserAuth, CredentialsAuth, TokenManagerAuth, MindConnectAgent) // to use different authorization types in Insights Hub or implement the TokenRotation // interface if you want to provide your own authorizer. // // The default constructor uses frontend authorization. const sdk = new MindSphereSdk(); const am = sdk.GetAssetManagementClient(); const assets = await am.GetAssets({ filter: JSON.stringify({ typeId: { startsWith: `${tenant}`, }, }), }); // you will get fully typed assets in response ``` If an API is missing (and you would like to contribute a client for it), take a look at [contributing to SDK](contributing-to-sdk.html). # TypeScript SDK - Authentication & Authorization The typescript SDK works with so called Authorizers which can be passed to the constructor of the client. The SDK comes with five authorizer implementations which support different deployment scenarios for your application. All authorizer implement the `TokenRotation` interface so that you can also develop your own custom authorizer class by implementing that interface. Info The `HttpAction` method is overridden in different authorizers, this is how the SDK works in both backend and frontend scenarios. All backend authorizer are automatically rotating the bearer tokens, you don't have to do it manually. For the frontend authorization we suggest to implement some kind of keep alive functionality in your app to avoid session expiration. ```javascript export interface TokenRotation { RenewToken(): Promise; GetToken(): Promise; GetGateway(): string; GetTenant(): string; HttpAction(....)} ``` ## BrowserAuth Info The `BrowserAuth` authentication and authorization only works for applications which are registered behind [Insights Hub APP Gateway](https://developer.mindsphere.io/concepts/concept-gateway-url-schemas.html) This Authorizer implements support for calling applications from frontend, where `Authorization: Bearer...` token is not required. Instead, the authorizer reuses the SESSION and XSRF-TOKEN Cookies and sets the X-XSRF-TOKEN header for you. - [Calling APIs from Frontend](https://developer.mindsphere.io/concepts/concept-authentication.html#calling-apis-from-frontend) - [Local Development with Session Cookies](https://developer.mindsphere.io/howto/howto-local-development.html#accessing-mindsphere-apis-using-session-cookies) The Authorizer uses Insights Hub Frontend API Calling Schema: ```text {web_app_host}/api/ ``` This means that it is using relative URLs as it is expecting to run in the browser behind the Insights Hub API gateway so it will call e.g. `/api/identitymanagement/v3/Users` instead of `https://gateway.eu1.mindsphere.io/api/identitymanagement/v3/Users`. You can simplify your local development by using the Insights Hub [development proxy](../cli/development-proxy.html) when using this authorizer. You need to redirect the `/api/**` calls of your application to use the the development proxy which is running on `http://localhost:7707`. ### Code Sample ```javascript // Browser authentication is the Default Authentication for the MindSphereSdk // it is used automatically if no other authorizer is passed const sdk = new MindSphereSdk(); // this is equivalent to new MindSphereSdk(new BrowserAuth()); const assetManagement = sdk.GetAssetManagementClient(); await assetManagement.GetAssets(); ``` The clients can also be instantiated separately (the `MindSphereSdk` class is just a convenience.) ```javascript const assetManagement = new AssetManagementClient(); await assetManagement.GetAssets(); ``` ## UserAuth Info The `UserAuth` authentication and authorization only works for applications which are registered behind [Insights Hub APP Gatewa](https://developer.mindsphere.io/concepts/concept-gateway-url-schemas.html) This Authorizer implements support for calling applications from backend, where `Authorization: Bearer...` Token *is required*. - [Calling APIs from Backend](https://developer.mindsphere.io/concepts/concept-authentication.html#calling-apis-from-backend) The Authorizer uses Insights Hub Backend API Calling Schema: ```text gateway.{region}.{mindsphere-domain}/api/... ``` You can simplify your local development by using the Insights Hub [development proxy](../cli/development-proxy.html) when using this authorizer, just replace the gateway url with `http://localhost:7707` when instantiating this authorizer. ### Code Sample Here is the class definition ```javascript export class UserAuth extends MindConnectBase implements TokenRotation { /** * Creates an instance of UserAuth. * extract token from http request headers (req.get("authorization")) * * @param {string} token * @param {string} gateway * * extract token from curreny http request: * Gateway has to follow the https://gateway.{region}.{mindsphere-domain} schema * * @see https://developer.mindsphere.io/concepts/concept-authentication.html#calling-apis-from-backend * * @memberOf UserAuth */ constructor(token: string, gateway: string) } ``` And here is an example ```javascript const bearerToken = request.get("Authorization"); const sdk = new MindSphereSdk(new UserAuth(bearerToken, "https://gateway.eu1.mindsphere.io")); const identity = sdk.GetIdentityManagementClient(); const newUser = await identity.PostUser({ userName: username }); ``` ## TokenManager Auth Info The `UserAuth` authentication and authorization only works for applications which are registered behind [Insights Hub APP Gatewa](https://developer.mindsphere.io/concepts/concept-gateway-url-schemas.html) and for self hosted applications which are just calling Insights Hub APIs. This Authorizer implements the bearer token rotation for app credentials. - [Self-Hosted API access](https://developer.mindsphere.io/howto/howto-selfhosted-api-access.html) - [Token Management Service](https://developer.mindsphere.io/apis/exchange-tokenmanager/api-tokenmanager-overview.html) The Authorizer uses Insights Hub Backend API Calling Schema: ```text gateway.{region}.{mindsphere-domain}/api/... ``` If you are using this type of credentials you can call Insights Hub APIs directly, there is no need to use development proxy. ### Code Sample Here is the class definition ```javascript export class TokenManagerAuth extends AuthBase implements TokenRotation { /** * Creates an instance of TokenManagerAuth. * @param {string} _gateway * @param {string} _basicAuth * @param {string} _hostTenant * @param {string} _userTenant * @param {string} _appName * @param {string} _appVersion * @memberOf TokenManagerAuth */ constructor( protected _gateway: string, protected _basicAuth: string, protected _hostTenant: string, protected _userTenant: string, protected _appName: string, protected _appVersion: string ); ``` The most important part is that you need to build the basic auth token using your Token Manager Login and Password. Encode the following combination of user name/ID and password/secret using Base64: ```text : ``` Build the Basic Auth token using the word Basic, followed by a space and the encoding result, e.g.: ```text Basic ZGlvcDEtaGVybWlvbmUtaGVybWlvbmU6c2RqaGZhc2RqaGZqYXNkaGZqa2FzZGh ``` - [Building the X-SPACE-AUTH-KEY](https://developer.mindsphere.io/apis/exchange-tokenmanager/api-tokenmanager-samples.html) ```javascript const basicAuth = `Basic ${Buffer.from("" + ":" + "").toString("base64")}`; const sdk = new MindSphereSdk( new TokenManagerAuth( "https://gateway.eu1.mindsphere.io", basicAuth, "hostTenant", "userTenant", "myApp", "1.0.0" )); const timeSeries = sdk.GetTimeSeriesClient(); await timeSeries.PutTimeSeries(assetid, aspectname, [{_time: new Date() rpm: "120"}] ) ``` ## CredentialAuth (deprecated) You can (almost always) just use TokenManagerAuth instead. Info The `CredentialsAuth` authentication and authorization only works for applications which are registered behind [Insights Hub APP Gatewa](https://developer.mindsphere.io/concepts/concept-gateway-url-schemas.html) and for self hosted applications which are just calling Insights Hub APIs. This Authorizer implements the bearer token rotation for app credentials. - [Self-Hosted API access](https://developer.mindsphere.io/howto/howto-selfhosted-api-access.html) - [Token Management Service](https://developer.mindsphere.io/apis/exchange-tokenmanager/api-tokenmanager-overview.html) The Authorizer uses Insights Hub Backend API Calling Schema: ```text gateway.{region}.{mindsphere-domain}/api/... ``` If you are using this type of credentials you can call Insights Hub APIs directly, there is no need to use development proxy. ### Code Sample Here is the class definition ```javascript export class CredentialAuth extends AuthBase implements TokenRotation { /** * Creates an instance of CredentialAuth. * @param {string} _gateway * @param {string} _basicAuth * @param {string} _tenant * * @memberOf CredentialAuth */ constructor(protected _gateway: string, protected _basicAuth: string, protected _tenant: string) ``` The most important part is that you need to build the basic auth token using your Token Manager Login and Password. Encode the following combination of user name/ID and password/secret using Base64: ```text : ``` Build the Basic Auth token using the word Basic, followed by a space and the encoding result, e.g.: ```text Basic ZGlvcDEtaGVybWlvbmUtaGVybWlvbmU6c2RqaGZhc2RqaGZqYXNkaGZqa2FzZGh ``` Example Using ServiceCredentials ```javascript const basicAuth = `Basic ${Buffer.from("" + ":" + "").toString("base64")}`; const sdk = new MindSphereSdk(new CredentialAuth("https://gateway.eu1.mindsphere.io", basicAuth, "hostTenant")); const timeSeries = sdk.GetTimeSeriesClient(); await timeSeries.PutTimeSeries(assetid, aspectname, [ { _time: new Date(), rpm: "120", humidity: "12", temperature: "12.5" }, ]); ``` ## Insights Hub Agent as Authorizer The Insights Hub agents implement the same `TokenRotation` interface like all other Authorizers which means that they can be used with SDK as well. (The agents will have a limited set of scopes but important AssetManagement methods will work) ```javascript const configuration = require("agentconfig.json"); const agent = new MindConnectAgent(configuration); agent.SetupAgentCertificate(fs.readFileSync("private.key")); await agent.OnBoard(); const sdk = new MindSphereSdk(agent); await sdk.GetAssetManagementClient().GetAssetType(assetTypeId, { exploded: true }); ``` ## Using credentials instead of Authorizer For TokenManagerAuth and CredentialAuth you can also use just the object with authentication credentials instead of the Authorizer class. The following two lines are equivalent: ```javascript const sdk = MindSphereSdk(new CredentialAuth("https://gateway.eu1.mindsphere.io", basicAuth, "hostTenant")); const sdk = MindSphereSdk({ gateway: "https://gateway.eu1.mindsphere.io", basicAuth: basicAuth, tenant: "hostTenant" }); ``` and the same is valid for these lines: ```javascript const sdk = new MindSphereSdk( new TokenManagerAuth("https://gateway.eu1.mindsphere.io", basicAuth, "hostTenant", "userTenant", "myApp", "1.0.0") ); const sdk = new MindSphereSdk({ gateway: "https://gateway.eu1.mindsphere.io", basicAuth: basicAuth, tenant: "hostTenant", usertenant: "userTenant", appName: "myApp", appVersion: "1.0.0", }); ``` # TypeScript SDK - Contributing If you want to contribute with a new client implementation to typescript SDK here are some guidelines: ## 1. Create an Issue Tell the community that you are working on an client :) ## 2. Create the file structure **client and models:** ```text \src\api\sdk\\ .models.ts .ts ``` **tests:** ```text \test\ .spec.ts ``` **cli command(s):** ```text \src\cli\commands folder ``` Just follow the existing code examples ;) ## 3. Generate the Models You can generate the models form OpenAPI specification using. e.g. and typescript-fetch template. The models belong into the namespace called ``Models ```javascript export namespace TimeSeriesAggregateModels { ....models go here } ``` ## 4. Implement the Client, Tests and CLI commands Implement the client for your API using `this.HttpAction method` and following the existing conventions. Using HttpAction ensures that the API Client works with different authorizers. Here is an example for TimeSeriesAggregateClient: ```javascript // Always extend SdkClient, this will give you access to this.HttpAction export class TimeSeriesAggregateClient extends SdkClient { // base Url from OpenAPI specification private _baseUrl: string = "/api/iottsaggregates/v3"; // the path parameters belong in the function parameters and the querystring parameters in params object public async GetAggregates( entityid: string, propertyset: string, params: { from: Date; to: Date; intervalValue: number; intervalUnit: string; select?: string } ): Promise { const qs = toQueryString(params); return (await this.HttpAction({ verb: "GET", gateway: this.GetGateway(), // always use this.GetGateway() and this.GetToken() authorization: await this.GetToken(), //this is overriden in different authorizers //and ensures that the Client works in frontend and in backend. baseUrl: `${this._baseUrl}/aggregates/${entityid}/${propertyset}?${qs}`, message: "GetTimeSeriesAggregates", // this is used for logging })) as TimeSeriesAggregateModels.Aggregates; // always cast to the real result type from models } } ``` You also have to create a Method for lazy loading of your new client on the `MindSphereSdk` class ```javascript private _timeSeriesAggregateClient?: TimeSeriesAggregateClient; public GetTimeSeriesAggregateClient(): TimeSeriesAggregateClient { this._timeSeriesAggregateClient = this._timeSeriesAggregateClient || new TimeSeriesAggregateClient(this._authenticator); return this._timeSeriesAggregateClient; } ``` and add export to `src\api\sdk\index.ts` file: ```javascript ... export * from "./iotaggregate/iot-timeseries-aggregate"; export * from "./iotaggregate/iot-timeseries-aggregate-models"; ... ``` Make sure that the tests are running and create a pull request. We would also really appreciate a CLI contribution. ♥️ # TypeScript SDK - HttpAction If a method is missing you can still call Insights Hub APIs using the HttpAction Method which is available on every Client. Take a look first at this implementation. You should always pass `this.GetGateway()`, and `await this.GetToken()` to the HttpAction, which makes sure that the authorizer works correctly. ```javascript export class TimeSeriesAggregateClient extends SdkClient { // base Url from OpenAPI specification private _baseUrl: string = "/api/iottsaggregates/v3"; // the path parameters belong in the function parameters and the querystring parameters in params object public async GetAggregates( entityid: string, propertyset: string, params: { from: Date; to: Date; intervalValue: number; intervalUnit: string; select?: string } ): Promise { const qs = toQueryString(params); return (await this.HttpAction({ verb: "GET", gateway: this.GetGateway(), // always use this.GetGateway() and this.GetToken() authorization: await this.GetToken(), //this is overriden in different authorizers //and ensures that the Client works in frontend and in backend. baseUrl: `${this._baseUrl}/aggregates/${entityid}/${propertyset}?${qs}`, message: "GetTimeSeriesAggregates", // this is used for logging })) as TimeSeriesAggregateModels.Aggregates; // always cast to the real result type from models } } ``` The full HttpAction method has more parameters, here is an explanation for the not so obvious ones: - `octetStream`: indicates that the response is an octetStream (file) - `multiPartFormData`: lets the method know that the content of message should be multipartFormData instead of JSON. - `rawResponse=true` will return `HttpResponse` instead of parsed and typed JSON message so that you can parse it on your own - `returnHeaders=true` will just return HttpHeaders of the http action - `ignoreCodes` list of HttpCodes for which the method will not throw an exception ```javascript /** * perform http action * * @param {({ * verb: "GET" | "POST" | "PATCH" | "PUT" | "DELETE"; * gateway: string; * baseUrl: string; * authorization: string; * body?: Object; * message?: string; * octetStream?: boolean; * multiPartFormData?: boolean; * additionalHeaders?: Object; * noResponse?: boolean; * rawResponse?: boolean; * returnHeaders?: boolean; * ignoreCodes?: number[]; * })} { * verb, * gateway, * baseUrl, * authorization, * body, * message, * octetStream, * multiPartFormData, * additionalHeaders, * noResponse, * rawResponse, * returnHeaders, * ignoreCodes, * } * @returns {Promise} * * @memberOf MindConnectBase */ public async HttpAction({ verb, gateway, baseUrl, authorization, body, message, octetStream, multiPartFormData, additionalHeaders, noResponse, rawResponse, returnHeaders, ignoreCodes, }: { verb: "GET" | "POST" | "PATCH" | "PUT" | "DELETE"; gateway: string; baseUrl: string; authorization: string; body?: Object; message?: string; octetStream?: boolean; multiPartFormData?: boolean; additionalHeaders?: Object; noResponse?: boolean; rawResponse?: boolean; returnHeaders?: boolean; ignoreCodes?: number[]; }): Promise ``` If you have implemented such method call please [consider contributing to the SDK](contributing-to-sdk.html). # Insights Hub and Industrial IoT Authentication Helper Chrome Extension This chrome extension can be used to simplify the development tasks for which you need to use the [SESSION and XSRF-TOKEN](https://developer.mindsphere.io/howto/howto-local-development.html#generate-user-credentials) to access Industrial IoT APIs. It provides an easy way to copy the Industrial IoT authentication cookies to the clipboard without having to go to chrome developer tools. ## Installation Go to [Chrome Web Store](https://chrome.google.com/webstore/detail/mindsphere-authentication/licndiiilobojikmhmmcgdbpmnmdeoee) chrome web store and click on **add to chrome**. Once the extension is installed: log in to your Insights Hub tenant, navigate to your application and click on the extension icon to display the available application cookies. You can now click e.g. on **session** or **XSRF** button which will copy the corresponding cookie to your clipboard. The cookies can be used with the tools like [curl](https://developer.mindsphere.io/howto/howto-local-development.html#curl-example), [postman](https://developer.mindsphere.io/howto/howto-local-development.html#postman-example_1) etc. (Click on the links for more examples) ### GET example ```bash #!/bin/bash curl -vv \ -G \ --cookie "SESSION=;XSRF-TOKEN=" \ -X GET \ https://--..mindsphere.io/api/assetmanagement/v3/assets ``` ### PUT example ```bash #!/bin/bash curl -v \ -X PUT https://--..mindsphere.io/api/ \ --cookie "SESSION=;XSRF-TOKEN=" \ --header "x-xsrf-token: " \ --header "Content-Type: application/json" \ --data-bin @ ``` ## Using the cookies with Insights Hub and Industrial IoT CLI The extension works really well with the Industrial IoT CLI in the [Session Cookie - XSRF-Token Configuration](../mindconnect-nodejs/cli/setting-up-the-cli.html) because the **Bash**, **PS**, and **CMD** buttons already copy the cookies in the required format for Industrial IoT CLI. Use the extension to copy the cookies to the clipboard, paste them to the command line and start using the [Command Line Interface (CLI)](../mindconnect-nodejs/cli/index.html) right away. ## Using the cookies with Insights Hub and Industrial IoT Development Proxy You can also use the extension to simplify the start of [MindSphere Development Proxy](../mindconnect-nodejs/cli/development-proxy.html) in a similar fashion. Just copy/paste the cookies to the command line and start the development proxy. This will start the development proxy with the credentials of the logged in users so that you can test the Industrial IoT authentication and authorization also in the local environment. **Important: Treat the authentication cookies like your credentials and use them only in secure environments.** # Insights Hub and Industrial IoT Development License Agreement December 2018 This Development License Agreement (the “License”) stipulates the rights that Siemens AG, Germany (“Siemens”) grants to you (“you”) in receipt of certain development material in relation to Insights Hub and Industrial IoT (the “Platform”). This material may include software, sample code, scripts, libraries, software development kits, technology, documentation, and other proprietary material or information (collectively, the “Licensed Material”). Siemens retains the right to utilize its affiliated companies in pursuing any of its rights and fulfilling any of its obligations under this License. Therefore, the term “Siemens” as used herein may also refer to affiliated companies that are directly or indirectly owned or controlled by the ultimate parent company of Siemens AG and who have been authorized by Siemens AG to distribute the Licensed Material. ## 1. Proprietary Rights in the Licensed Material All rights, title, interest and know-how in and to the Licensed Material, any part and improvement thereof and all intellectual property rights in or to the foregoing, other than those rights expressly granted in this License, shall remain wholly vested in Siemens or its third party business partners and/or licensors. ## 2. License Grant If you have entered a separate commercial license agreement with Siemens that covers the Licensed Material, such license agreement shall prevail and this License shall not apply. You must accept this License to use the Licensed Material. If you use the Licensed Material, any such use shall constitute acceptance of this License. The Licensed Material is licensed, not sold. Subject to the terms and conditions of this License, Siemens grants you a non-exclusive, worldwide, royalty-free, non-transferable right to: - publish, modify and create derivative works of small source code examples (no longer than 50 lines of code) contained in Licensed Material, - integrate and publish Licensed Material in software that interoperates with the Platform, e.g. by transmission of data between applications, devices, systems or equipment and the Platform (hereinafter referred as “Customizations”), and license such Customizations to third parties to enable your offering of services relating to the Platform, - use, run and copy Licensed Material for your own internal use, and - sublicense Licensed Material to a service provider for the purposes of this section. You shall not: - use the Licensed Material for any other technology than the Platform, or - use the Licensed Material to abuse the Platform, or to compromise its integrity or security, or - disassemble or reverse-engineer any Licenses Material provided in object-code format, unless such action is permitted by an applicable Open Source license or the law that applies to you, or - remove or modify Siemens copyright, patent, trademark or attribution notices in the Licensed Material. ## 3. Feedback If you provide Siemens with feedback regarding the Licensed Material, you grant Siemens a non-exclusive, worldwide, royalty-free and perpetual right to use, copy, modify, distribute, and sublicense any such feedback in any way. ## 4. Open Source Software The Licensed Material may contain third-party software, including Open Source Software (“OSS”). If required, Siemens will furnish a notice file with the Licensed Material. To the extent the licenses terms applicable to such OSS are in contradiction to this License, the OSS license conditions shall prevail. ## 5. Responsibility for Use of the Licensed Material and Customizations Except any liability directly caused by Siemens violating its obligations under this License, you are responsible for the use of the Licensed Material (and all consequences arising therefrom), regardless of whether the use is undertaken by you, your employees or any third parties. You will ensure that the use of the Licensed Material by you, your employees or your customers complies with your obligations under this License. Should you become aware of any violation of your obligations under this License, you will (i) immediately inform Siemens thereof in reasonable detail and (ii) cease to use the Licensed Material. You are responsible for (i) properly integrating the Licensed Material into your Customizations, (ii) configuring and testing any Customization and making sure that a Customization is able to connect to and/or interoperate with the Platform, (iii) regular testing and monitoring of integrity, accuracy and timeliness of data transmission through a Customization (e.g. by monitoring over the Platform), (iv) the security of a Customization, and (v) the security of your and your customer’s system, any third party system or any data stored on such systems. You are responsible for the integrity, accuracy and timeliness of the exchange of data between a Customization and the Platform. Nothing in this License shall in any way constitute or be communicated by you or your customers to be an endorsement from us of a Customization in which you implemented the Licensed Material. ## 6. Warranty, Liability and Indemnification SIEMENS PROVIDES THE LICENSED MATERIAL FREE OF CHARGE AND THEREFORE “AS IS” WITHOUT ANY EXPRESS OR IMPLIED WARRANTIES. ANY STATEMENTS OR REPRESENTATIONS ABOUT THE LICENSED MATERIAL AND ITS RESPECTIVE FUNCTIONALITY IN THE LICENSED MATERIAL CONSTITUTE MERELY TECHNICAL INFORMATION AND ARE NEITHER A GUARANTEE NOR A WARRANTY. SIEMENS IS NOT AWARE THAT THE LICENSED MATERIAL VIOLATES ANY THIRD PARTY RIGHTS IF USED IN ACCORDANCE WITH THE LICENSE. FURTHERMORE, SIEMENS DOES NOT WARRANT THAT THE LICENSED MATERIAL IS FAIL-SAFE, FAULT-TOLERANT, ERROR-FREE OR THAT ANY DATA TRANSMITTED THROUGH THE LICENSED MATERIAL OR A CUSTOMIZATION WILL BE SECURE OR NOT OTHERWISE LOST OR DAMAGED. Siemens disclaims all liability for all damages of any kind irrespective of the legal ground in relation to the Licensed Material. However, nothing in this License shall exclude or limit Siemens’ liability in case of death or personal injury of any person, or in case of our willful misconduct, gross negligence, fraud or fraudulent misrepresentation. Any limitations of liability set forth in this License shall also apply for the benefit of Siemens’ subcontractors, employees, directors, agents or any other person acting for Siemens. You agree to indemnify, defend, and hold Siemens harmless from and against all losses, expenses, damages and costs, including reasonable attorneys’ fees, resulting from or arising out of any violation of this License by you or any other person you provided with the Licensed Material, regardless of your knowledge, including any claim, proceeding, action, fine, loss, cost and damages arising out of or relating to any non-compliance with export control regulations. ## 7. Term and Termination, Updates If Siemens reasonably determines that you are breaching this License and you fail to remediate such breach without undue delay, Siemens may suspend or terminate your Platform access temporarily or permanently and terminate this License at its sole discretion. Siemens may, at its sole discretion, make available updates or security patches for the Licensed Material. Outdated versions of the Licensed Material may not be able to connect to and/or interoperate with the Platform. ## 8. Applicable Law This License shall be governed by and construed in accordance with the laws of Switzerland, without giving effect to any choice-of-law rules that may require the application of the law of another jurisdiction. The UN Convention on Contracts for the International Sale of Goods shall not apply. # MindConnect Node-RED Node Overview ## Introduction A Node-RED node which can be used to upload the time series data, files and events to Insights Hub via MindConnect. The node also has support for the Industrial Data Lake. It runs on x86, Raspberry PI, Siemens IOT 2040 and is also available as a docker container for your convenience. - [GitHub](https://github.com/mindsphere/node-red-contrib-mindconnect) - [npm](https://www.npmjs.com/package/@mindconnect/node-red-contrib-mindconnect) The node is written in typescript/javascript without any native dependencies so it should work beside x86 also on other platforms (e.g. on raspberry pi, [Simatic IoT2000 Gateways](https://www.siemens.com/global/en/products/automation/pc-based/iot-gateways.html), [Siemens Industrial Edge](../../industrial-edge/overview.html) etc. You just need [Node-RED](https://nodered.org). ## About MindConnect Node-RED Node The Node-RED platform targets the makers who want an easy, flow-based way to configure and manage the flow of data with their IoT devices. This node provides the connectivity to Insights Hub for TimeSeries upload, File upload and Event Creation. ## Using the Mindconnect Node-RED Node Take a look at the [MindConnect Node-RED Playground](https://playground.mindconnect.rocks). # MindConnect Node-RED Node - Getting Started ## Installing the node The node is available at the npmjs.org registry and can be installed with help of the following command: ```bash # change to your ~./node-red/ folder cd ~/.node-red/ npm install @mindconnect/node-red-contrib-mindconnect ``` Info - install to the .node-red folder if you have installed node-red globally - install to the userDir directory if you have custom userDir - make sure that your nodejs version is relatively current ## Node-RED - Manage Palette Installation You can install the node also via the Manage palette feature in the Node-RED administration UI. ## How to use the Node-RED node Since version 3.9.0 it is possible to completely configure the agent from Node-RED. You will only need the initial Boarding configuration from the Insights Hub UI. ### Step 0: Create (at least) one asset and one agent in Insights Hub - Create an asset in Asset Manager for your data - Create an agent of the type MindConnectLib [core.mclib] and store the agent. ### Step 1: Get the initial agent configuration from Insights Hub Asset Manager You can choose between: - **RSA_3072** public/private key pair (3072bit) for enhanced security which requires more computing power on the devices and - **SHARED_SECRET** shared key (256bit) for lightweight devices. If you want to use RSA_3072 you will have to create a 3072bit key for your device, e.g.. with `openssl`: ```bash openssl genrsa -out private.key 3072 ``` There is no additional configuration required for SHARED_SECRET security profile. ### Step 2: Copy the agent onboarding information (and if necessary the RSA 3072 private key) to the node and deploy the flow Copy the agent onboarding information and optionally the RSA_3072 private key to the node and deploy the flow. ### Step 3: Press the agent configuration button and select the target asset The most common agent configuration setup is to have a 1:1 mapping between the Node-RED agent which is delivering the data and your target MindSphere Asset. If this type of configuration is sufficient for your use case you just have to click on the asset to which you want to map the data in the asset list. (you can use the filter asset listbox to quickly find your asset) The node will automatically configure all necessary data sources and mapping for you. If you need a more complex setup, just click on the **MindSphere Configuration Dialog** button which will lead you to the configuration dialog in the Insights Hub, where you can create more complex configurations and mappings. ### Step 4: Create and deploy the flow and send data You can use the node to send timeseries, bulk timeseries, events and files to Insights Hub. The templates for the input messages are listed below, but you can also just use the **Agent Information** button which will let you copy the corresponding template to clipboard. #### Send data points The node requires json objects as input in following format (e.g. from a function node) ```javascript const values = [ { dataPointId: "1000000000", qualityCode: "1", value: "42" }, { dataPointId: "1000000001", qualityCode: "1", value: "33.7" }, { dataPointId: "1000000003", qualityCode: "1", value: "45.76" }, ]; msg._time = new Date(); msg.payload = values; return msg; ``` The node will validate if the data is valid for your agent configuration. his feature can be switched off in the settings but it is not recommended to do so. #### Send data points in bulk The node requires json objects as input in following format (e.g. from a function node if you want to use bulk upload) ```javascript const values = [ { timestamp: "2018-11-09T07:46:36.699Z", values: [ { dataPointId: "1000000000", qualityCode: "1", value: "42" }, { dataPointId: "1000000001", qualityCode: "1", value: "33.7" }, { dataPointId: "1000000003", qualityCode: "1", value: "45.76" }, ], }, { timestamp: "2018-11-08T07:46:36.699Z", values: [ { dataPointId: "1000000000", qualityCode: "1", value: "12" }, { dataPointId: "1000000001", qualityCode: "1", value: "13.7" }, { dataPointId: "1000000003", qualityCode: "1", value: "15.76" }, ], }, ]; msg.payload = values; return msg; ``` **Note:** All Insights Hub timestamps must be in the **ISO format** (use `toISOString()` function). #### Send events The node requires json objects as input in following format (e.g. from a function node). You can send an event to any asset you have access to in your tenant. Just use the asset id in the entityid. ```javascript msg.payload = { entityId: "d72262e71ea0470eb9f880176b888938", // optional, use assetid if you want to send event somewhere else :) sourceType: "Agent", sourceId: "application", source: "Meowz", severity: 30, // 0-99 : 20:error, 30:warning, 40: information description: "Event sent at " + new Date().toISOString(), timestamp: new Date().toISOString(), additionalproperty1: "123", additionalproperty2: "456", }; return msg; ``` If you are using the custom events instead of Insights Hub Standard Events please include the following switch in the message. ```javascript msg._customEvent = true; ``` #### File Upload The node requires json objects as input in following format (e.g. from a function node). You can upload file to any asset you have access to in your tenant. Just use the asset id in the entityid. ```javascript msg.payload = { entityId: "d72262e71ea0470eb9f880176b888938", //optional (per default files are uploaded to the agent) fileName: "digitaltwin.png", // you can also pass an instance of a Buffer fileType: "image/png", //optional, it is automatically determined if there is no fileType specified filePath: "images/digitaltwin.png", // required if you are using buffer instead of the file name description: "testfile", }; return msg; ``` If the experimental chunking feature is on, the files which are larger than 8MB will be uploaded in 8 MB Chunks. #### Data Lake File Upload Precondition for data lake upload is that [MindSphere Integrated Data Lake](https://www.dex.siemens.com/mindsphere/applications/integrated-data-lake) is purchased and write-enabled. The node requires json objects as input in following format (e.g. from a function node). ```javascript // take a look at the flow examples at https://playground.mindconnect.rocks // // Preconditions : data-lake is purchased and enabled for writing (see mc data-lake --mode write CLI command) // // Agents can only upload files to a path which is prefixed with their agent id // The MindConnect Node will apply this prefix automatically to the dataLakeFileUpload Path // You can pass either a javascript buffer or path to file in the dataLakeFile property for upload // The subTenantId can be optionally added to the messsage const dataLakeFileInfo = { dataLakeFile: "my/path/to/file.txt", dataLakeFilePath: "uploads/file.txt", }; // Uncomment the next code line if you just want to generate an upload url (in msg._signedUrl) // without actually uploading the file // msg._ignorePayload = true; msg.payload = dataLakeFileInfo; return msg; ``` Please note: - Agents can only upload files to a path which is prefixed with their agent id - The MindConnect Node will apply this prefix automatically to the dataLakeFileUpload Path - You can pass either a javascript buffer or path to file in the dataLakeFile property for upload - The subTenantId can be optionally added to the message #### Reading Asset Information You can read the data (e.g. static asset variables, or full asset information) from Insights Hub using the following message. This can be used to implement a "digital shadow/digital twin" pattern, where the change in the Insights Hub variables is reflected to the real world asset. See [bidirectional communication example flow](https://playground.mindconnect.rocks/#flow/9ff72be.3d502d8) on playground for a full example. ```javascript msg.payload = { assetId: "{assetId}", includeShared: false, propertyNames: [], }; return msg; ``` You can reduce the number of items in payload by specifying list of properties to include in the message: e.g. `propertyNames: ["variables"] or ["location"]` #### Executing custom functions using Insights Hub javascript/typescript SDK The node can be used to execute a complex script which uses [MindSphere javascript/typescript SDK](../mindconnect-nodejs/sdk/index.html). The node will create an asyncronous function with one parameter (sdk) and the specified function body and execute it. You can only call the Insights Hub APIs which allow agent authorization. ```javascript msg.payload = { function: ` const assetManagement = sdk.GetAssetManagementClient(); const asset = await assetManagement.GetAsset('{assetId}'); return asset; `, }; return msg; ``` #### Error handling in the flows The node can be configured to retry all Insights Hub operations (1-10 times, with delay of time * 300ms before the next try) If you need more complex flows, the node also returns the ```javascript msg._mindsphereStatus; // OK on success othervise error msg._error; // The timestamped error message ``` properties which can be used to create more complex flows. (e.g. in the flow below, the unrecoverable errors are written in error.log file and the failed data is stored in `backupdata.log` file) ## JWT Token Generation for SouthBound APIs The node can be used to generate authentication tokens which you can use to call your own custom southbound APIs. The msg.headers will have a Insights Hub Authorization JWT. ```javascript msg._includeMindSphereToken = true; ``` if you just want to get the token without sending any data to Insights Hub ```javascript msg._ignorePayload = true; ``` Treat tokens as you would any other credentials. ## Demo flows Node RED playground [Demo Flows](https://playground.mindconnect.rocks) [MindConnect Node-RED playground](https://playground.mindconnect.rocks) provides following demo flows importing following data points to Insights Hub - CPU-Usage - Batched MQTT Data - OPCUA Data - Real Weather Data to Insights Hub - Simulated Water Pump Data ## Securing API Endpoints The corresponding API calls for reading the data source configuration and mappings in Agent Configuration and Agent Information dialog require that the user has: `mindconnect.read` permission. The automatic configuration requires `mindconnect.write` permission. ## Troubleshooting If you have problems with your agent: 1. Stop the agent. 1. Move or delete the content of the .mc folder (the json files with configuration and authentication settings). 1. Offboard the agent. 1. Create new settings for the mindconnect library. 1. copy the new settings to the node. ### Resetting the agent settings from version 3.7.0 Since version 3.7.0. it is possible to delete the content of the .mc/agentconfig.json file and the agent settings directly from the node. Press on the "delete local configuration" button on the node, confirm the dialog and redeploy the node. If you are having problems, it is a good idea to restart the Node-RED runtime completely. ### Diagnostic in Insights Hub If the data is not arriving in your configured asset you should take a look if the data is being dropped in Insights Hub because of a misconfiguration. The agent diagnostic button will lead you directly to the agent diagnostic application in the Insights Hub. ## Generating the documentation You can always generate the current HTML documentation by running the command below. ```bash #this generates a docs/ folder the with full documentation of the library. npm run doc ``` ## Proxy support Set the http_proxy or HTTP_PROXY environment variable if you need to connect via proxy. ```bash # set http proxy environment variable if you are using e.g. fiddler on the localhost. export HTTP_PROXY=http://localhost:8888 ``` ## How to setup development environment ```bash # create a directory ../devnodes cd .. mkdir devnodes cd devnodes # this registers your development directory with node red npm link ../node-red-contrib-mindconnect # after that in you can start developing with cd ../node-red-contrib-mindconnect npm run start-dev # your node red flows will be stored in the ../devnodes directory ``` # MindConnect-Node-RED Node - License MIT License Copyright © 2019 Siemens AG Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. # Insights Hub and Industrial IoT documentation # Contact [Contact our experts](https://plm.sw.siemens.com/en-US/contact-plm/) or [sign up for free](https://plm.sw.siemens.com/en-US/insights-hub/start/) to start your industrial IoT journey today. Learn how to get the full potential of your data step by step with Insights Hub. # Insights Hub Developer Documentation ## Get the basics [Familiarize yourself](https://documentation.mindsphere.io/MindSphere/index.html) with the architecture, building blocks, concepts, and interactions, from token exchange to access control. ## Connect devices Connect (almost any) Industrial device to harvest data and remotely connect to your on-site assets. [Here's how](https://documentation.mindsphere.io/MindSphere/index.html). ## Explore System Tools With our [system tools](https://documentation.mindsphere.io/MindSphere/system-tools/overview.html), you manage your environment every step of the way, from your first venture into IoT to running fully digitalized business models. ## Build your own Apps and Solutions Here are your building blocks! To help you develop and - if you so choose - share and monetize your own app, we've put together the latest concepts, [How-tos, SDKs, and APIs](https://documentation.mindsphere.io/MindSphere/app-development/overview.html). ## Developer Resources Access the full [developer documentation](https://developer.mindsphere.io) for Insights Hub and Industrial IoT. ## Contact [Contact our experts](https://plm.sw.siemens.com/en-US/contact-plm/) or [sign up for free](https://plm.sw.siemens.com/en-US/insights-hub/start/) to start your industrial IoT journey today. Learn how to get the full potential of your data step by step with Insights Hub. | Link | Description | | ---------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | [Developer Documentation](https://documentation.mindsphere.io/MindSphere/app-development/overview.html) | Developer Documentation. | | [App Development Concepts](https://documentation.mindsphere.io/MindSphere/concepts/concept-app-development.html) | API Lifecycle, Versioning, Authentication and authorization concepts. | | [APIs and Services](docs/apis/index.html) | List of Insights Hub APIs and Services. | | [Insights Hub Resources](https://documentation.mindsphere.io/MindSphere/resources/index.html) | List of all published resources for Insights Hub Developers. | | [DevOps Cockpit](https://documentation.mindsphere.io/MindSphere/apps/devops-cockpit/introduction.html) | DevOps Cockpit offers customers one centralized place to bring your apps into Insights Hub. | | [DevOps Guide](https://documentation.mindsphere.io/MindSphere/apps/devops-guide/introduction-scope.html) | DevOps Guide provides information for the development and testing of applications, as well as for deployment, productive operation and provisioning of applications via the respective Account and/or Environment. | | [How Tos and Tutorials](https://documentation.mindsphere.io/MindSphere/howto/howto-local-development.html) | Accessing Industrial IoT APIs during local development. | | [GitHub](https://github.com/mindsphere) | Insights Hub and Industrial IoT GitHub | - [Overview](overview.html) - [Developer Documentation](documentation.html) - [Insights Hub and Industrial IoT APIs](/apis.html?productLine=Insights+Hub+and+Industrial+IoT&resourceType=AsyncAPI%2COpenAPI) - [Insights Hub and Industrial IoT SDKs and Libraries](/apis.html?productLine=Insights+Hub+and+Industrial+IoT&resourceType=Library%2CSDK) - [Getting Started](docs/apis/index.html) - Platform Core - Identity Management Service - [Overview](docs/apis/core-identitymanagement/api-identitymanagement-overview.html) - APIs & References - [API Specification](docs/apis/core-identitymanagement/api-identitymanagement-api.html) - Resource Access Management - [Overview](docs/apis/core-resourceaccessmanagement/api-resourceaccessmanagement-overview.html) - Concepts - [Definitions Structure](docs/apis/core-resourceaccessmanagement/concept-policy.html) - [Understanding Policy effects](docs/apis/core-resourceaccessmanagement/concept-enforcement.html) - [ABAC based Policiy Conditions](docs/apis/core-resourceaccessmanagement/ABAC-based-policy.html) - Quickstarts - [Create and Assign a Policy - Policy Manager](docs/apis/core-resourceaccessmanagement/quick-policy-manager.html) - [Assign a Policy - REST](docs/apis/core-resourceaccessmanagement/quick-rest.html) - Troubleshoot - [Common Issues](docs/apis/core-resourceaccessmanagement/common-issues.html) - [Known Issues](docs/apis/core-resourceaccessmanagement/known-issues.html) - References - [Administration API Specification](docs/apis/core-resourceaccessmanagement/api-resourceaccessmanagement-api.html) - [Action List](docs/apis/core-resourceaccessmanagement/api-resourceaccessmanagement-actions-list.html) - [Services & Required Actions](docs/apis/core-resourceaccessmanagement/api-resourceaccessmanagement-services-actions-map.html) - [Limits](docs/apis/core-resourceaccessmanagement/limits.html) - Message Broker Service - [Overview](docs/apis/core-messagebroker/api-messagebroker-overview.html) - [Topics](docs/apis/core-messagebroker/api-messagebroker-topics.html) - APIs & References - [API Specification](docs/apis/core-messagebroker/api-messagebroker-api.html) - [Samples](docs/apis/core-messagebroker/api-messagebroker-samples.html) - OAuth Authorization Server API - [Overview](docs/apis/core-oauthauthorizationserver/api-oauthauthorizationserver-overview.html) - APIs & References - [API Specification](docs/apis/core-oauthauthorizationserver/api-oauthauthorizationserver-api.html) - Tenant Management Service - [Overview](docs/apis/core-tenantmanagement/api-tenantmanagement-overview.html) - APIs & References - [API Specification](docs/apis/core-tenantmanagement/api-tenantmanagement-api.html) - Token Management Service - [Overview](docs/apis/exchange-tokenmanager/api-tokenmanager-overview.html) - APIs & References - [API Specification](docs/apis/exchange-tokenmanager/api-tokenmanager-api.html) - [Samples](docs/apis/exchange-tokenmanager/api-tokenmanager-samples.html) - Usage Transparency Service - [Overview](docs/apis/core-usagetransparency/api-usagetransparency-overview.html) - APIs & References - [API Specification](docs/apis/core-usagetransparency/api-usagetransparency-api.html) - [Samples](docs/apis/core-usagetransparency/api-usagetransparency-samples.html) - IoT and Storage - IoT File Service - [Version 4.x - What's New?](docs/apis/iot-iotfile/api-iotfile-version-v4-whats-new.html) - [Overview](docs/apis/iot-iotfile/api-iotfile-overview.html) - APIs & References - [Filtering](docs/apis/iot-iotfile/api-iotfile-references-filtering.html) - [Multi Part Operations](docs/apis/iot-iotfile/api-iotfile-references-multipart.html) - [Bulk Delete Operation](docs/apis/iot-iotfile/api-iotfile-bulkdeletion-samples.html) - [API Specification](docs/apis/iot-iotfile/api-iotfile-api.html) - IoT Time Series Service - IoT Time Series Service API - [Overview](docs/apis/iot-iottimeseries/api-iottimeseries-overview.html) - [Ingest Rate Calculator](docs/apis/iot-iottimeseries/api-ingestratecalculator-overview.html) - APIs & References - [API rate limits](docs/apis/iot-iottimeseries/api-iottimeseries-apiratelimits.html) - [API Specification](docs/apis/iot-iottimeseries/api-iottimeseries-api.html) - [Samples](docs/apis/iot-iottimeseries/api-iottimeseries-samples.html) - IoT Time Series Subscription Notification Async API - [Overview](docs/apis/iot-iottimeseries/api-subscription-notification-overview.html) - APIs & References - [Async API Specification](docs/apis/iot-iottimeseries/api-iottimeseries-async-api.html) - [Async API - Europe 1](docs/apis/iot-iottimeseries/api-iottimeseries-async-api-swagger-3-0-1.html) - IoT TS Aggregates Service - [Version 4.x - What's New?](docs/apis/iot-iottsaggregates/api-iottsaggregates-version-v4-whats-new.html) - [IoT TS Aggregates Service - V3.x to V4.x Migration Guide](docs/apis/iot-iottsaggregates/api-iottsaggregates-v3-v4-migration-guide.html) - Overview - [Overview for v3.x](docs/apis/iot-iottsaggregates/api-iottsaggregates-overview.html) - [Overview for v4.x](docs/apis/iot-iottsaggregates/api-iottsaggregates-overview-v4.html) - Basics - [Request Parameters for v3.x](docs/apis/iot-iottsaggregates/api-iottsaggregates-basics-parameters.html) - [Request Parameters for v4.x](docs/apis/iot-iottsaggregates/api-iottsaggregates-basics-parameters-v4.html) - APIs & References - [Selecting](docs/apis/iot-iottsaggregates/api-iottsaggregates-references-select.html) - [API Specification](docs/apis/iot-iottsaggregates/api-iottsaggregates-api.html) - Samples - [Aggregate v3.x Samples](docs/apis/iot-iottsaggregates/api-iottsaggregates-samples.html) - [Aggregate v4.x Samples](docs/apis/iot-iottsaggregates/api-iottsaggregates-samples-v4.html) - IoT TS Bulk Service - [Overview](docs/apis/iot-iottsbulk/api-iottsbulk-overview.html) - APIs & References - [API Specification](docs/apis/iot-iottsbulk/api-iottsbulk-api.html) - [Samples](docs/apis/iot-iottsbulk/api-iottsbulk-samples.html) - IoT Time Series Stream Registration Service - [Overview](docs/apis/iot-iottsstream/api-timeseries-streaming-overview.html) - APIs & References - [API Specification](docs/apis/iot-iottsstream/api-iottsstreamregistration-api.html) - Integrated Data Lake Service - [Version 4.x - What's new?](docs/apis/iot-integrated-data-lake/api-integrated-data-lake-version-v4-whats-new.html) - [Overview](docs/apis/iot-integrated-data-lake/api-integrated-data-lake-overview.html) - [Overview for v4.x](docs/apis/iot-integrated-data-lake/api-integrated-data-lake-overview-version4.html) - APIs & References - [API Specification](docs/apis/iot-integrated-data-lake/api-integrated-data-lake-api.html) - Samples - [Upload Data](docs/apis/iot-integrated-data-lake/api-integrated-data-lake-samples-upload-data.html) - [Download Data](docs/apis/iot-integrated-data-lake/api-integrated-data-lake-samples-download-data.html) - [Config Events](docs/apis/iot-integrated-data-lake/api-integrated-data-lake-samples-config-events.html) - [Metadata](docs/apis/iot-integrated-data-lake/api-integrated-data-lake-samples-metadata.html) - [Metadata Management](docs/apis/iot-integrated-data-lake/api-integrated-data-lake-samples-metadata-management.html) - [Cross Account Accesses](docs/apis/iot-integrated-data-lake/api-integrated-data-lake-samples-working-with-cross-account-accesses.html) - [Metadata & Object Search](docs/apis/iot-integrated-data-lake/api-integrated-data-lake-samples-metadata-object-search.html) - [Metadata Search](docs/apis/iot-integrated-data-lake/api-integrated-data-lake-samples-metadata-search.html) - [Toggle Searchable](docs/apis/iot-integrated-data-lake/api-integrated-data-lake-samples-toggle-searchable.html) - [Bulk Batch File Upload](docs/apis/iot-integrated-data-lake/api-integrated-data-lake-samples-bulk-batch-file-upload.html) - [Delete Multiple Objects](docs/apis/iot-integrated-data-lake/api-integrated-data-lake-samples-delete-multiple-objects.html) - [Time Series Import](docs/apis/iot-integrated-data-lake/api-integrated-data-lake-samples-time-series-import.html) - [Object and Folder Operations](docs/apis/iot-integrated-data-lake/api-integrated-data-lake-object-and-folder-operation.html) - [Upload Large Files](docs/apis/iot-integrated-data-lake/api-integrated-data-lake-upload-large-file.html) - Connectivity - Agent Management Service - Agent Management API - [Overview](docs/apis/connectivity-agentmanagement/api-agentmanagement-overview.html) - APIs & References - [API Specification](docs/apis/connectivity-agentmanagement/api-agentmanagement-api.html) - [Usage quota and limits](docs/apis/connectivity-agentmanagement/api-agentmanagement-apiratelimits.html) - Agent Management Async API - [Overview](docs/apis/connectivity-agentmanagement-async/api-agentmanagement-async-overview.html) - APIs & References - [Async API Specification](docs/apis/connectivity-agentmanagement-async/api-agentmanagement-async-api.html) - [Async API - Private Cloud](docs/apis/connectivity-agentmanagement-async/api-agentmanagement-async-api-swagger-rancher.html) - [Async API - Europe 1](docs/apis/connectivity-agentmanagement-async/api-agentmanagement-async-api-swagger-v3-0-0-eu1.html) - MindConnect - MindConnect API - [Overview](docs/apis/connectivity-mindconnect/api-mindconnect-overview.html) - APIs & References - [API Specification](docs/apis/connectivity-mindconnect/api-mindconnect-api.html) - [Filtering](docs/apis/connectivity-mindconnect/api-mindconnect-references-filtering.html) - [API Rate Limits](docs/apis/connectivity-mindconnect/api-mindconnect-apiratelimits.html) - MindConnect Async API - [Overview](docs/apis/connectivity-mindconnect-async/api-mindconnect-async-overview.html) - APIs & References - [Async API Specification](docs/apis/connectivity-mindconnect-async/api-mindconnect-async-api.html) - [Async API - Europe 1](docs/apis/connectivity-mindconnect-async/api-mindconnect-async-api-swagger-v3-1-0-eu1.html) - [Async API - Private Cloud](docs/apis/connectivity-mindconnect-async/api-mindconnect-async-api-swagger-v3-0-0.html) - [Samples](docs/apis/connectivity-mindconnect-async/api-mindconnect-async-sample.html) - Custom Data Services API - [Overview](docs/apis/connectivity-customdataservices/api-customdataservices-overview.html) - APIs & References - [API Specification](docs/apis/connectivity-customdataservices/api-customdataservices-api.html) - MindConnect MQTT API - [Overview](docs/apis/connectivity-mindconnect-mqtt/api-mindconnect-mqtt-overview.html) - APIs & References - [API Specification](docs/apis/connectivity-mindconnect-mqtt/api-mindconnect-mqtt-api.html) - [API Rate Limits](docs/apis/connectivity-mindconnect-mqtt/api-mindconnect-mqtt-apiratelimits.html) - Asset Modeler Async API - [Overview](docs/apis/connectivity-assetmodeler/api-assetmodeler-overview.html) - APIs & References - [Async API Specification](docs/apis/connectivity-assetmodeler/api-assetmodeler-api.html) - [Async API v4.0.0 - Private Cloud](docs/apis/connectivity-assetmodeler/api-assetmodeler-async-api-swagger-v4-0-0-rancher.html) - [Async API v3.0.0 - Private Cloud](docs/apis/connectivity-assetmodeler/api-assetmodeler-async-api-swagger-v3-0-0.html) - [Async API v4.0.0 - Europe 1](docs/apis/connectivity-assetmodeler/api-assetmodeler-async-api-swagger-v4-0-0-eu1.html) - [Async API v3.1.0 - Europe 1](docs/apis/connectivity-assetmodeler/api-assetmodeler-async-api-swagger-v3-1-0-eu1.html) - [Samples](docs/apis/connectivity-assetmodeler/api-assetmodeler-samples.html) - Commanding Feature - [Overview](docs/apis/connectivity-commanding/api-commanding-overview.html) - APIs & References - [API Specification](docs/apis/connectivity-commanding/api-commanding-api.html) - [Async API - Europe 1](docs/apis/connectivity-commanding/api-commanding-asyncapi-swagger-3-1-1-eu1.html) - [Async API - Private Cloud](docs/apis/connectivity-commanding/api-commanding-asyncapi-swagger-3-1-1-rancher.html) - [API - Europe 1](docs/apis/connectivity-commanding/api-commanding-api-swagger-3-3-0-eu1.html) - Samples - [Samples for subscribing to an MQTT topic](docs/apis/connectivity-commanding/api-commanding-subscribing-sample.html) - [Samples for Publishing MQTT commands to agents](docs/apis/connectivity-commanding/api-commanding-publishing-sample.html) - Remote Services - [Overview](docs/apis/connectivity-remoteservices/api-remote-services-overview.html) - APIs & References - [API Specification](docs/apis/connectivity-remoteservices/api-remote-services-api.html) - Advanced - Asset Management Service - Asset Management Service API - [Overview](docs/apis/advanced-assetmanagement/api-assetmanagement-overview.html) - APIs & References - [Basic Types](docs/apis/advanced-assetmanagement/api-assetmanagement-references-basictypes.html) - [Filtering](docs/apis/advanced-assetmanagement/api-assetmanagement-references-filtering.html) - [Pagination](docs/apis/advanced-assetmanagement/api-assetmanagement-references-pagination.html) - [API Rate Limits](docs/apis/advanced-assetmanagement/api-assetmanagement-apiratelimits.html) - [API Specification](docs/apis/advanced-assetmanagement/api-assetmanagement-api.html) - Samples - [Modeling Assets](docs/apis/advanced-assetmanagement/api-assetmanagement-samples-modelingassets.html) - [Modeling Hierarchical Asset Structures](docs/apis/advanced-assetmanagement/api-assetmanagement-samples-breadcrumbs.html) - [Using Default Values](docs/apis/advanced-assetmanagement/api-assetmanagement-samples-defaultvalues.html) - [Update Asset Type](docs/apis/advanced-assetmanagement/api-assetmanagement-samples-updatingtypes.html) - Asset Management Subscription Notification Async API - [Overview](docs/apis/advanced-assetmanagement/api-assetmanagement-subscription-notification-overview.html) - APIs & References - [Async API Specification](docs/apis/advanced-assetmanagement/api-assetmanagement-async-api.html) - [Async API - Europe 1](docs/apis/advanced-assetmanagement/api-assetmanagement-async-api-3-0-0.html) - [Samples](docs/apis/advanced-assetmanagement/async-api-assetmanagement-samples-assets.html) - Event Management Service - Event Management Service API - [Overview](docs/apis/advanced-eventmanagement/api-eventmanagement-overview.html) - [Event Count Estimator](docs/apis/advanced-eventmanagement/api-eventmanagement-eventestimator-overview.html) - APIs & References - [Filtering](docs/apis/advanced-eventmanagement/api-eventmanagement-references-filtering.html) - [API Rate Limits](docs/apis/advanced-eventmanagement/api-eventmanagement-apiratelimits.html) - [API Specification](docs/apis/advanced-eventmanagement/api-eventmanagement-api.html) - Samples - [Best Practices](docs/apis/advanced-eventmanagement/api-eventmanagement-best-practices.html) - [Event Operations](docs/apis/advanced-eventmanagement/api-eventmanagement-samples-events.html) - [Event Type Operations](docs/apis/advanced-eventmanagement/api-eventmanagement-samples-event-types.html) - Event Management Subscription Notification Async API - [Overview](docs/apis/advanced-eventmanagement/api-eventmanagement-subscription-notification-overview.html) - APIs & References - [Async API Specification](docs/apis/advanced-eventmanagement/api-eventmanagement-async-api.html) - [Async API v3.0.1](docs/apis/advanced-eventmanagement/api-eventmanagement-async-api-3-0-1.html) - [Samples](docs/apis/advanced-eventmanagement/async-api-eventmanagement-samples-events.html) - Notification Service - [Overview](docs/apis/advanced-notification/api-notification-overview.html) - APIs & References - [API Specification](docs/apis/advanced-notification/api-notification-api.html) - Samples - [Samples for v3.x](docs/apis/advanced-notification/api-notification-samples.html) - [Samples for v4.x](docs/apis/advanced-notification/api-mobile-notification-samples.html) - Case Management Service - [Overview](docs/apis/advanced-casemanagement/api-casemanagement-overview.html) - APIs & References - [Filtering](docs/apis/advanced-casemanagement/api-casemanagement-filtering.html) - [Pagination](docs/apis/advanced-casemanagement/api-casemanagement-pagination.html) - [API Specification](docs/apis/advanced-casemanagement/api-casemanagement-api.html) - [Samples](docs/apis/advanced-casemanagement/api-casemanagement-samples.html) - Visual Flow Creator - [Overview](docs/apis/advanced-visual-flow-creator/api-visual-flow-creator-overview.html) - APIs & References - [API Specification](docs/apis/advanced-visual-flow-creator/api-visual-flow-creator-api.html) - Rules Management - [Overview](docs/apis/advanced-rulesmanagement/api-rulesmanagement-overview.html) - APIs & References - [API Specification](docs/apis/advanced-rulesmanagement/api-rulesmanagement-api.html) - Opcenter Intelligence - [Overview](docs/apis/advanced-opcenter-intelligence/api-opcenter-intelligence-overview.html) - APIs & References - [API Specification](docs/apis/advanced-opcenter-intelligence/api-opcenter-intelligence-api.html) - Traceability - [Overview](docs/apis/advanced-traceability/api-traceability-overview.html) - APIs & References - [API Specification](docs/apis/advanced-traceability/api-traceability-api.html) - Analytics Services - Data Exchange Service - [Overview](docs/apis/analytics-dataexchange/api-dataexchange-overview.html) - APIs & References - [API Rate Limits](docs/apis/analytics-dataexchange/api-dataexchange-apiratelimits.html) - [API Specification](docs/apis/analytics-dataexchange/api-dataexchange-api.html) - [Samples](docs/apis/analytics-dataexchange/api-dataexchange-samples.html) - Job Manager Service - [Overview](docs/apis/analytics-jobmanager/api-jobmanager-overview.html) - APIs & References - [API Rate Limits](docs/apis/analytics-jobmanager/api-jobmanager-apiratelimits.html) - [API Specification](docs/apis/analytics-jobmanager/api-jobmanager-api.html) - [Samples](docs/apis/analytics-jobmanager/api-jobmanager-samples.html) - Model Management Service - [Overview](docs/apis/analytics-modelmanagement/api-modelmanagement-overview.html) - APIs & References - [API Rate Limits](docs/apis/analytics-modelmanagement/api-modelmanagement-apiratelimits.html) - [API Specification](docs/apis/analytics-modelmanagement/api-modelmanagement-api.html) - [Samples](docs/apis/analytics-modelmanagement/api-modelmanagement-samples.html) - Resources - [Index](docs/resources/index.html) - Insights Hub Monitor Plugin - [Developer Documentation](docs/resources/insights-hub-monitor-plugin-sdk/index.html) - [Getting started](docs/resources/insights-hub-monitor-plugin-sdk/insights-hub-monitor-plugin-gettingstarted.html) - [References](docs/resources/insights-hub-monitor-plugin-sdk/insights-hub-monitor-plugin-references.html) - [Demo Plugin](docs/resources/insights-hub-monitor-plugin-sdk/insights-hub-monitor-plugin-demo.html) - [Checklist](docs/resources/insights-hub-monitor-plugin-sdk/insights-hub-monitor-plugin-requirements.html) - [FAQ](docs/resources/insights-hub-monitor-plugin-sdk/insights-hub-monitor-plugin-faq.html) - [Change Log](docs/resources/insights-hub-monitor-plugin-sdk/changelog.html) - MindConnect Library v3 - [Developer Documentation](docs/resources/mindconnect-lib/resources-mclib-overview.html) - Getting started - [Linux](docs/resources/mindconnect-lib/resources-mclib-getting-started.html) - [Windows](docs/resources/mindconnect-lib/resources-mclib-getting-started-win.html) - [Interfaces and References](docs/resources/mindconnect-lib/resources-mclib-references.html) - [Samples](docs/resources/mindconnect-lib/resources-mclib-samples.html) - [Certificate](docs/resources/mindconnect-lib/resources-mclib-cacert.html) - [Change Log](docs/resources/mindconnect-lib/resources-mclib-change-log.html) - MindConnect Library v4 - [Developer Documentation](docs/resources/mindconnect-lib-v4/resources-mclib-overview.html) - [Dependencies](docs/resources/mindconnect-lib-v4/resources-mclib-dependencies.html) - [Replacing MCL Modules](docs/resources/mindconnect-lib-v4/resources-mclib-replacing-mcl-modules.html) - [Building MCL](docs/resources/mindconnect-lib-v4/resources-mclib-building-mcl.html) - [Agent Application](docs/resources/mindconnect-lib-v4/resources-mclib-agent-app.html) - [Examples](docs/resources/mindconnect-lib-v4/resources-mclib-examples.html) - [Reference Document](docs/resources/mindconnect-lib-v4/resources-mclib-reference.html) - [Sony Spresense Guide](docs/resources/mindconnect-lib-v4/resources-mclib-porting-sony-spresense.html) - [Migrating From 3.x To 4.x](docs/resources/mindconnect-lib-v4/resources-mclib-migration-guide.html) - [Certificate](docs/resources/mindconnect-lib-v4/resources-mclib-cacert.html) - [Change Log](docs/resources/mindconnect-lib-v4/resources-mclib-change-log.html) - Industrial IoT SDK for Java - [Developer Documentation](docs/resources/industrial-iot-sdk-java/index.html) - [Getting started](docs/resources/industrial-iot-sdk-java/sdkreadme.html) - [Exception Handling](docs/resources/industrial-iot-sdk-java/exception_handling.html) - [Logging](docs/resources/industrial-iot-sdk-java/sdk_logging.html) - [API clients and References](docs/resources/industrial-iot-sdk-java/javadocs/index.html) - Code samples - [Asset Management](docs/resources/industrial-iot-sdk-java/apidocs/AssetManagement.html) - [Event Analytics](docs/resources/industrial-iot-sdk-java/apidocs/EventAnalytics.html) - [Event Management](docs/resources/industrial-iot-sdk-java/apidocs/EventManagement.html) - [IoT File](docs/resources/industrial-iot-sdk-java/apidocs/IOT_File.html) - [IoT Time Series](docs/resources/industrial-iot-sdk-java/apidocs/IOT_Timeseries.html) - [IoT TS Aggregate](docs/resources/industrial-iot-sdk-java/apidocs/IOT_Timeseries_Aggregate.html) - Industrial IoT SDK V2 for Java - [Developer Documentation](docs/resources/industrial-iot-sdk-java-v2/index.html) - [Getting started](docs/resources/industrial-iot-sdk-java-v2/sdkreadme_v2.html) - [Xcelerator Adoption in SDK & Platform](docs/resources/industrial-iot-sdk-java-v2/xcelerator_sdk_adoption.html) - [Exception Handling](docs/resources/industrial-iot-sdk-java-v2/exception_handling_v2.html) - [Token Handling](docs/resources/industrial-iot-sdk-java-v2/token_handling_v2.html) - [Logging](docs/resources/industrial-iot-sdk-java-v2/sdk_logging_v2.html) - [API clients and References](docs/resources/industrial-iot-sdk-java-v2/javadocs-v2/index.html) - Code samples - [Agent Management](docs/resources/industrial-iot-sdk-java-v2/apidocs/AgentManagement.html) - [Asset Management](docs/resources/industrial-iot-sdk-java-v2/apidocs/AssetManagement.html) - [Event Management](docs/resources/industrial-iot-sdk-java-v2/apidocs/EventManagement.html) - [IoT File](docs/resources/industrial-iot-sdk-java-v2/apidocs/IOT_File.html) - [IoT Time Series](docs/resources/industrial-iot-sdk-java-v2/apidocs/IOT_TimeseriesV.html) - [IoT TS Aggregate](docs/resources/industrial-iot-sdk-java-v2/apidocs/IOT_Aggregate.html) - [IoT TS Aggregate V4.X](docs/resources/industrial-iot-sdk-java-v2/apidocs/TSAggregateV4.html) - [Insights Hub Credentials](docs/resources/industrial-iot-sdk-java-v2/apidocs/Credentials.html) - [Token Management](docs/resources/industrial-iot-sdk-java-v2/apidocs/TokenManager.html) - [MindConnect](docs/resources/industrial-iot-sdk-java-v2/apidocs/MindConnect.html) - [IoT Time Series](docs/resources/industrial-iot-sdk-java-v2/apidocs/IOT_Timeseries.html) - [IOT Bulk](docs/resources/industrial-iot-sdk-java-v2/apidocs/IOT_Bulk.html) - [IoT TS Aggregate V4.X](docs/resources/industrial-iot-sdk-java-v2/apidocs/TSAggregateV4.html) - [Integrated Data Lake](docs/resources/industrial-iot-sdk-java-v2/apidocs/IDL.html) - Industrial IoT SDK for Node.js - [Developer Documentation](docs/resources/industrial-iot-sdk-node/index.html) - [Getting started](docs/resources/industrial-iot-sdk-node/nodereadme.html) - [Xcelerator Adoption in SDK & Platform](docs/resources/industrial-iot-sdk-node/xcelerator_sdk_adoption.html) - [Token Handling](docs/resources/industrial-iot-sdk-node/token_handling.html) - [Logging](docs/resources/industrial-iot-sdk-node/logging.html) - [API clients and References](docs/resources/industrial-iot-sdk-node/jsdoc/index.html) - Code samples - [Asset Management](docs/resources/industrial-iot-sdk-node/apidocs/AssetManagement.html) - [IoT File](docs/resources/industrial-iot-sdk-node/apidocs/IOT_File.html) - [IoT Time Series](docs/resources/industrial-iot-sdk-node/apidocs/IOT_Timeseries.html) - [IoT TS Aggregate](docs/resources/industrial-iot-sdk-node/apidocs/IOT_Aggregate.html) - [IoT TS Aggregates Client for node](docs/resources/industrial-iot-sdk-node/apidocs/TsAggregatesV4.html) - [Insights Hub Credentials](docs/resources/industrial-iot-sdk-node/apidocs/Credentials.html) - Industrial IoT SDK for Python - [Developer Documentation](docs/resources/industrial-iot-sdk-python/index.html) - [Getting started](docs/resources/industrial-iot-sdk-python/pyreadme.html) - [Xcelerator Adoption in SDK & Platform](docs/resources/industrial-iot-sdk-python/xcelerator_sdk_adoption.html) - [Error Handling](docs/resources/industrial-iot-sdk-python/error_handling.html) - [Token Handling](docs/resources/industrial-iot-sdk-python/token_handling.html) - [Logging](docs/resources/industrial-iot-sdk-python/logging.html) - [API clients and References](docs/resources/industrial-iot-sdk-python/sphinxdoc/index.html) - Code samples - [Asset Management](docs/resources/industrial-iot-sdk-python/apidocs/AssetManagement.html) - [IoT Time Series\*](docs/resources/industrial-iot-sdk-python/apidocs/IOT_Timeseries.html) - [IoT TS Aggregate\*](docs/resources/industrial-iot-sdk-python/apidocs/IOT_Aggregate.html) - [IoT TS Aggregate V4.X\*](docs/resources/industrial-iot-sdk-python/apidocs/IOT_TsAggregatesV4.html) - [IoT File](docs/resources/industrial-iot-sdk-python/apidocs/IOT_File.html) - [IoT Time Series](docs/resources/industrial-iot-sdk-python/apidocs/IOT_TimeSeriesV.html) - [IoT TS Bulk](docs/resources/industrial-iot-sdk-python/apidocs/IOT_TSBulk.html) - [MindConnect](docs/resources/industrial-iot-sdk-python/apidocs/MindConnect.html) - [Insights Hub Credentials](docs/resources/industrial-iot-sdk-python/apidocs/Credentials.html) - [Data Contextualization](docs/resources/industrial-iot-sdk-python/apidocs/SDI.html) - [Integrated Data Lake](docs/resources/industrial-iot-sdk-python/apidocs/IDL.html) - [Contact](contact.html) # Insights Hub and Industrial IoT [Insights Hub](https://plm.sw.siemens.com/en-US/insights-hub/) drives smart manufacturing through the industrial Internet of Things. Gain actionable insights with asset and operational data and improve your processes. ## Why Insights Hub? Deliver business value with industrial IoT data by implementing reliable asset monitoring, enhancing manufacturing performance and efficiency and enabling quality prediction and much more. Make improved operational and business decisions with data-driven insights. ### Explore Data Connect your assets to the cloud, collect and explore your data, and strategically develop your IoT capabilities. ### Discover Insights Unlock a world of previously unattainable insights: use intelligent analytic tools to better understand and improve processes. ### Transform Business Transform your business, processes and products at scale. Create a competitive advantage, reduce costs and improve quality across the entire product lifecycle and supply chain. Insights Hub is transforming the world of smart manufacturing, and operations and setting new standards of productivity and sustainability with industrial IoT proven solutions. \[!\[ \](https://video.sw.cdn.siemens.com/mgc/thumbnails/e9671880-3974-403f-a7e5-a8d9e77b7fa3-en-US-thumbnail.png)\](https://video.sw.cdn.siemens.com/mgc/videos/1200/e9671880-3974-403f-a7e5-a8d9e77b7fa3-en-US-video.mp4) ## Industrial Operations X [Industrial Operations X](https://www.siemens.com/global/en/products/automation/topic-areas/industrial-operations-x.html) is accelerating the transformation from automated to adaptive production. Our Industrial Operations X portfolio combines multiple tools to deliver powerful IoT results. It helps you—across all industries—accelerate production processes by making them adaptive, people-centric and holistically integrated. With this new approach to operations and smart manufacturing, you are better equipped to face ever-changing industry demands, business models and workforces. As part of the Industrial Operations X portfolio, Insights Hub delivers business value from IoT data, while the combined IoT capabilities of Industrial Operations X enables customers, partners and organizations across the Siemens Xcelerator ecosystem to build industry-specific applications. # Insights Hub Services - Index In this section you will find all currently published services that can be used. This section will be updated regularly as we continuously enrich our ecosystem. Refer to the concepts section for information on the [API Versioning](https://documentation.mindsphere.io/MindSphere/concepts/concept-api-versioning.html) and the [API Lifecycle](https://documentation.mindsphere.io/MindSphere/concepts/concept-api-lifecycle.html) in Insights Hub. Important If you want to continue using Insights Hub APIs, refer to [APIs & Services IH](index.html) section. Insights Hub APIs are either HTTP-based or messaging-based. HTTP-based APIs come with an [OpenAPI specification](https://github.com/OAI/OpenAPI-Specification/), describing the HTTP operations offered by a service to be called by clients. Messaging-based APIs come with an [AsyncAPI specification](https://github.com/asyncapi/asyncapi), describing the topics and messages offered by a service to either subscribe to or publish to by a client. The following tables list all available Insights Hub services with a description and the available API versions in the respective regions. By default, API versions are HTTP-based APIs. In case an AsyncAPI is provided the version is annotated accordingly. Warning Do not set the `ContentType` header of GET requests to `application/json` if the request body is empty. Many firewalls block this as it is identified as invalid request. Note Insights Hub will adhere to the API request URL validations in the upcoming release. This is a step to make the environment more secure. The API request URLs will be validated based on the use of trailing slash characters in requests pointing to dedicated resources. Please refer to the list below which contains examples of incorrect URLs and correct URLs. This will be applicable to all APIs in Insights Hub. | Examples of malformed requests | Examples of well-formed requests | | ------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------- | | GET `https://gateway.{region}.{mindsphere-domain}/api/iottsaggregates/v3/aggregates/{assetId}/{aspectName}/` | GET `https://gateway.{region}.{mindsphere-domain}/api/iottsaggregates/v3/aggregates/{assetId}/{aspectName}` | | GET `https://{tenantName}-assetmanagement.{region}.{mindsphere-domain}/api/assetmanagement/v3/assets/{id}/?includeShared=true` | GET `https://{tenantName}-assetmanagement.{region}.{mindsphere-domain}/api/assetmanagement/v3/assets/{id}?includeShared=true` | | PUT `https://{tenantName}-assetmanagement.{region}.{mindsphere-domain}/api/assetmanagement/v3/assettypes/{id}/` | PUT `https://{tenantName}-assetmanagement.{region}.{mindsphere-domain}/api/assetmanagement/v3/assettypes/{id}` | ## Platform Core | Service | Description | Europe 1 | | ------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------- | | [Identity Management](core-identitymanagement/api-identitymanagement-overview.html) | Use Customer Management APIs to manage your customers (e.g. add new ones or change existing) for own tenants only. | [3.3.3](core-identitymanagement/api-identitymanagement-api.html) | | [Resource Access Management](core-resourceaccessmanagement/api-resourceaccessmanagement-overview.html) | Use Resource Access Management APIs to manage your policies (e.g. add new ones or change existing) for own tenants only. | [3.4.0](core-resourceaccessmanagement/api-resourceaccessmanagement-api.html) | | [Message Broker](core-messagebroker/api-messagebroker-overview.html) | Use Message Broker APIs to enable asynchronous communication in the platform. | [3.0.1](core-messagebroker/api-messagebroker-api.html) | | [OAuth Authorization Server](core-oauthauthorizationserver/api-oauthauthorizationserver-overview.html) | Use OAuth Authorization Server to request authentication and authorization. | [3.2.0](core-oauthauthorizationserver/api-oauthauthorizationserver-api.html) | | [Tenant Management](core-tenantmanagement/api-tenantmanagement-overview.html) | Use Tenant Management to manage: subtenants, tenant information and legal information. | [4.7.0](core-tenantmanagement/api-tenantmanagement-api.html) | | [Token Management](exchange-tokenmanager/api-tokenmanager-overview.html) | The Token Manager API is used to provide an application access to data of other tenants, who use the application. | [3.0.1](exchange-tokenmanager/api-tokenmanager-api.html) | | [Usage Transparency Service](core-usagetransparency/api-usagetransparency-overview.html) | Usage Transparency Service offers a UI giving insight on your resource consumption on the Insights Hub environment. For Developers it as offers an API to track a metric defined by the developers. This metric can also be retrieved via an API of UTS. | [3.2.0](core-usagetransparency/api-usagetransparency-api.html) | ## IoT and Storage | Service | Description | Europe 1 | | ---------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------- | | [IoT File](iot-iotfile/api-iotfile-overview.html) | File Service to read, write and delete files: upload, update and delete files associated to assets; store metadata information, and search for files by metadata. | [4.1.0](iot-iotfile/api-iotfile-api.html) [3.7.0](iot-iotfile/api-iotfile-api.html) | | [IoT Time Series](iot-iottimeseries/api-iottimeseries-overview.html) | Use Time Series to create, read, update and delete dynamic data. Since time series data are always related to an asset, the instance of an asset must have been created by you beforehand. | [3.8.0](iot-iottimeseries/api-iottimeseries-api.html) | | [IoT Time Series Subscription Notification](iot-iottimeseries/api-subscription-notification-overview.html) | Get the notification for latest arrival of timeseries data. | [3.1.0](iot-iottimeseries/api-iottimeseries-async-api.html) | | [IoT TS Aggregates](iot-iottsaggregates/api-iottsaggregates-overview.html) | Use Time Series Aggregates Service to read aggregated time series values. Retrieve the following aggregated values per interval: Count, Sum, Average, Minimum, Maximum, First Value, Last Value,Standard Deviation. | [3.6.0](iot-iottsaggregates/api-iottsaggregates-api.html) [4.4.0](iot-iottsaggregates/api-iottsaggregates-api.html) | | [IoT TS Bulk](iot-iottsbulk/api-iottsbulk-overview.html) | Use Time Series Bulk Service to import and read high frequency time series data. | [3.9.0](iot-iottsbulk/api-iottsbulk-api.html) | | [IoT TS Stream Registration](iot-iottsstream/api-timeseries-streaming-overview.html) | Use Time Series Streaming Service to stream timeseries data to external streaming component. | - | | [Integrated Data Lake](iot-integrated-data-lake/api-integrated-data-lake-overview.html) | Integrated Data Lake is an application in Insights Hub to import and store the historical IoT data, access cross accounts and perform analytics on the data. | [3.14.0](iot-integrated-data-lake/api-integrated-data-lake-api.html) | ## Connectivity | Service | Description | Europe 1 | | -------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- | | [Agent Management](connectivity-agentmanagement/api-agentmanagement-overview.html) | Use Agent Management to create, edit or remove MindConnect elements, on-board and off board agents and set relations to assets. | [3.5.2](connectivity-agentmanagement/api-agentmanagement-api.html) | | [Agent Management Async API](connectivity-agentmanagement-async/api-agentmanagement-async-overview.html) | Use Agent Management to acquire an agent access token via MQTT agents. | [3.0.0](connectivity-agentmanagement-async/api-agentmanagement-async-api.html) | | [MindConnect API](connectivity-mindconnect/api-mindconnect-overview.html) | Use MindConnect API to develop custom agents and connect your device or application to Insights Hub. | [3.7.0](connectivity-mindconnect/api-mindconnect-api.html) | | [MindConnect Async API](connectivity-mindconnect-async/api-mindconnect-async-overview.html) | Use MindConnect Async API to ingest timeseries data. | [3.1.0](connectivity-mindconnect-async/api-mindconnect-async-api.html) | | [Custom Data Service API](connectivity-customdataservices/api-customdataservices-overview.html) | Use Custom Data Service API to register and manage custom topic mappings for data integration within Insights Hub. | [1.0.1](connectivity-customdataservices/api-customdataservices-api.html) | | [MindConnect MQTT API](connectivity-mindconnect-mqtt/api-mindconnect-mqtt-overview.html) | Use MindConnect MQTT API to manage the certificates for securely connecting the MQTT agents with Insights Hub. | [3.0.2](connectivity-mindconnect-mqtt/api-mindconnect-mqtt-api.html) | | [Asset Modeler Async API](connectivity-assetmodeler/api-assetmodeler-overview.html) | Asset Modeler Async API to to create assets, types, and mappings. | [3.1.0](connectivity-assetmodeler/api-assetmodeler-api.html) | | [Commanding Feature](connectivity-commanding/api-commanding-overview.html) | Use Commanding Feature to send command and/or data associated with the command up to 20 MindConnect MQTT agents. | [3.1.1](connectivity-commanding/api-commanding-api.html) (Async API) [3.3.0](connectivity-commanding/api-commanding-api.html) | | [Remote Services API](connectivity-remoteservices/api-remote-services-overview.html) | Use Remote Services API to facilitate secure connectivity to devices on the shopfloor, by means of Fine Grained Authorization Control (FGAC) | [3.0.0](connectivity-remoteservices/api-remote-services-api.html) | ## Advanced Services | Service | Description | Europe 1 | | ---------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------- | | [Asset Management](advanced-assetmanagement/api-assetmanagement-overview.html) | Represent physical assets from your site in Insights Hub. Use models and create instances, set relations to others and create structures such as hierarchies. | [3.19.0](advanced-assetmanagement/api-assetmanagement-api.html) | | [Asset Management Subscription Notification](advanced-assetmanagement/api-assetmanagement-subscription-notification-overview.html) | Get the notification for changes to Asset Model | [3.0.0](advanced-assetmanagement/api-assetmanagement-async-api.html) | | [Event Management](advanced-eventmanagement/api-eventmanagement-overview.html) | Manage standardized and customized events. Acquire events from the field & other applications. | [3.17.0](advanced-eventmanagement/api-eventmanagement-api.html) | | [Event Management Subscription Notification](advanced-eventmanagement/api-eventmanagement-subscription-notification-overview.html) | Get the notification for latest creation or updation of Event. | [3.1.0](advanced-eventmanagement/api-eventmanagement-async-api.html) | | [Notification Service](advanced-notification/api-notification-overview.html) | Use APIs or graphical user interface to send information to your users & customers via e-mail, sms or push/scheduled notification. | [4.4.2](advanced-notification/api-notification-api.html) [3.4.1](advanced-notification/api-notification-api.html) | | [Case Management Service](advanced-casemanagement/api-casemanagement-api.html) | Provide a basic digital workflow for work requests (maintenance, repair, inspection and incident handling) that are essential to monitor asset health and to detect technical issues before they lead to asset failure and downtime. | [3.5.0](advanced-casemanagement/api-casemanagement-api.html) | | [Visual Flow Creator](advanced-visual-flow-creator/api-visual-flow-creator-overview.html) | Visual Flow Creator API service can be used to create, update, retrieve, delete and other functionalities for the data flows | [3.0.0](advanced-visual-flow-creator/api-visual-flow-creator-api.html) | | [Rules Management](advanced-rulesmanagement/api-rulesmanagement-overview.html) | Rules Management API service can be used for configuring, reading, deleting and updating rules. | [4.0.0](advanced-rulesmanagement/api-rulesmanagement-api.html) | | [Opcenter Intelligence](advanced-opcenter-intelligence/api-opcenter-intelligence-overview.html) | Opcenter Intelligence API Service can be used to access Opcenter Intelligence manufacturing data warehouse data to perform analysis and KPI calculation within the supported Insights Hub Services. | [1.0.0](advanced-opcenter-intelligence/api-opcenter-intelligence-api.html) | | [Traceability](advanced-traceability/api-traceability-overview.html) | Traceability APIs support the application’s containment and release functionalities. They enable integration with external systems by providing access to headers and items that are either contained or released. Additionally, two dedicated APIs handle the acknowledgment action, which confirms the reading of containment or release requests. | [1.0.0](advanced-traceability/api-traceability-api.html) | ## Analytics Services | Service | Description | Europe 1 | | ------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------- | | [Data Exchange](analytics-dataexchange/api-dataexchange-overview.html) | The Data Exchange Service provides remote file storage and management support through simple REST API calls. The service offers support for uploading, downloading, renaming, and publishing within the same tenant files or folders. | [3.0.2](analytics-dataexchange/api-dataexchange-api.html) | | [Job Manager](analytics-jobmanager/api-jobmanager-overview.html) | The Job Manager Service allows customers to execute Zeppelin Notebooks using a scheduling mechanism, also providing full support for the execution environment, cleanup, storage and library support required to run the models. | [3.2.0](analytics-jobmanager/api-jobmanager-api.html) | | [Model Management](analytics-modelmanagement/api-modelmanagement-overview.html) | The Model Management Service provides analytical model management file storage with versioning support. The service provides help in maintaining version, dependency, parameters and authorship information. | [3.2.0](analytics-modelmanagement/api-modelmanagement-api.html) | # Asset Management Service – API Overview ## Region Europe 1 [Download OpenAPI Specification](/insights-hub/api_specs/assetmanagement-v3-19-0-xcelerator.yaml) # Asset Management - Usage quota and limits Asset Management APIs are restricted with following limits. | API Endpoint | Throttling/Restriction Criteria | XS | S | M | L | XL | | -------------------------------------------- | ------------------------------- | ----- | ----- | ----- | ----- | ----- | | Get aspecttypes | Requests in 1 Minute | 1485 | 1485 | 1485 | 2228 | 2970 | | Get aspecttypes | Requests in 1 Hour | 17820 | 17820 | 17820 | 26730 | 35640 | | Delete aspecttype/{id} | Requests in 1 Minute | 446 | 446 | 446 | 669 | 891 | | Delete aspecttype/{id} | Requests in 1 Hour | 1485 | 1485 | 1485 | 2228 | 2970 | | Get aspecttype/{id} | Requests in 1 Minute | 750 | 750 | 750 | 1125 | 1500 | | Get aspecttype/{id} | Requests in 1 Hour | 17820 | 17820 | 17820 | 26730 | 35640 | | Patch aspecttype/{id} | Requests in 1 Minute | 446 | 446 | 446 | 669 | 891 | | Patch aspecttype/{id} | Requests in 1 Hour | 1485 | 1485 | 1485 | 2228 | 2970 | | Put aspecttype/{id} | Requests in 1 Minute | 743 | 743 | 743 | 1114 | 1485 | | Put aspecttype/{id} | Requests in 1 Hour | 1485 | 1485 | 1485 | 2228 | 2970 | | Get assets | Requests in 1 Minute | 4000 | 4000 | 4000 | 4455 | 5940 | | Get assets | Requests in 1 Hour | 17820 | 17820 | 17820 | 26730 | 35640 | | Post assets | Requests in 1 Minute | 594 | 594 | 594 | 891 | 1188 | | Post assets | Requests in 1 Hour | 1485 | 1485 | 1485 | 2228 | 2970 | | Delete assets/{id} | Requests in 1 Minute | 446 | 446 | 446 | 669 | 891 | | Delete assets/{id} | Requests in 1 Hour | 1485 | 1485 | 1485 | 2228 | 2970 | | Get assets/{id} | Requests in 1 Minute | 4000 | 4000 | 4000 | 4455 | 5940 | | Get assets/{id} | Requests in 1 Hour | 17820 | 17820 | 17820 | 26730 | 35640 | | Patch assets/{id} | Requests in 1 Minute | 675 | 675 | 675 | 1013 | 1350 | | Patch assets/{id} | Requests in 1 Hour | 1875 | 1875 | 1875 | 2813 | 3750 | | Put assets/{id} | Requests in 1 Minute | 446 | 446 | 446 | 669 | 891 | | Put assets/{id} | Requests in 1 Hour | 1485 | 1485 | 1485 | 2228 | 2970 | | Get assets/{id}/aspects | Requests in 1 Minute | 2970 | 2970 | 2970 | 4455 | 5940 | | Get assets/{id}/aspects | Requests in 1 Hour | 17820 | 17820 | 17820 | 26730 | 35640 | | Delete assets/{id}/fileAssignments/{key} | Requests in 1 Minute | 891 | 891 | 891 | 1337 | 1782 | | Delete assets/{id}/fileAssignments/{key} | Requests in 1 Hour | 1485 | 1485 | 1485 | 2228 | 2970 | | Put assets/{id}/fileAssignments/{key} | Requests in 1 Minute | 891 | 891 | 891 | 1337 | 1782 | | Put assets/{id}/fileAssignments/{key} | Requests in 1 Hour | 1485 | 1485 | 1485 | 2228 | 2970 | | Delete assets/{id}/location | Requests in 1 Minute | 446 | 446 | 446 | 669 | 891 | | Delete assets/{id}/location | Requests in 1 Hour | 1485 | 1485 | 1485 | 2228 | 2970 | | Put assets/{id}/location | Requests in 1 Minute | 446 | 446 | 446 | 669 | 891 | | Put assets/{id}/location | Requests in 1 Hour | 1485 | 1485 | 1485 | 2228 | 2970 | | Post assets/{id}/move | Requests in 1 Minute | 446 | 446 | 446 | 669 | 891 | | Post assets/{id}/move | Requests in 1 Hour | 1485 | 1485 | 1485 | 2228 | 2970 | | Get assets/{id}/variables | Requests in 1 Minute | 1485 | 1485 | 1485 | 2228 | 2970 | | Get assets/{id}/variables | Requests in 1 Hour | 17820 | 17820 | 17820 | 26730 | 35640 | | Get assets/root | Requests in 1 Minute | 743 | 743 | 743 | 1114 | 1485 | | Get assets/root | Requests in 1 Hour | 17820 | 17820 | 17820 | 26730 | 35640 | | Get assettypes | Requests in 1 Minute | 3500 | 3500 | 3500 | 4455 | 5940 | | Get assettypes | Requests in 1 Hour | 17820 | 17820 | 17820 | 26730 | 35640 | | Delete assettypes/{id} | Requests in 1 Minute | 446 | 446 | 446 | 669 | 891 | | Delete assettypes/{id} | Requests in 1 Hour | 1485 | 1485 | 1485 | 2228 | 2970 | | Get assettypes/{id} | Requests in 1 Minute | 3500 | 3500 | 3500 | 4455 | 5940 | | Get assettypes/{id} | Requests in 1 Hour | 17820 | 17820 | 17820 | 26730 | 35640 | | Patch assettypes/{id} | Requests in 1 Minute | 446 | 446 | 446 | 669 | 891 | | Patch assettypes/{id} | Requests in 1 Hour | 1485 | 1485 | 1485 | 2228 | 2970 | | Put assettypes/{id} | Requests in 1 Minute | 743 | 743 | 743 | 1114 | 1485 | | Put assettypes/{id} | Requests in 1 Hour | 1485 | 1485 | 1485 | 2228 | 2970 | | Delete assettypes/{id}/fileAssignments/{key} | Requests in 1 Minute | 891 | 891 | 891 | 1337 | 1782 | | Delete assettypes/{id}/fileAssignments/{key} | Requests in 1 Hour | 1485 | 1485 | 1485 | 2228 | 2970 | | Put assettypes/{id}/fileAssignments/{key} | Requests in 1 Minute | 891 | 891 | 891 | 1337 | 1782 | | Put assettypes/{id}/fileAssignments/{key} | Requests in 1 Hour | 1485 | 1485 | 1485 | 2228 | 2970 | | Patch assettypes/{id}/variables | Requests in 1 Minute | 743 | 743 | 743 | 1114 | 1485 | | Patch assettypes/{id}/variables | Requests in 1 Hour | 1485 | 1485 | 1485 | 2228 | 2970 | | Get files | Requests in 1 Minute | 929 | 929 | 929 | 1393 | 1857 | | Get files | Requests in 1 Hour | 1485 | 1485 | 1485 | 2228 | 2970 | | Post files | Requests in 1 Minute | 224 | 224 | 224 | 336 | 447 | | Post files | Requests in 1 Hour | 372 | 372 | 372 | 558 | 743 | | Delete files/{fileid} | Requests in 1 Minute | 560 | 560 | 560 | 840 | 1119 | | Delete files/{fileid} | Requests in 1 Hour | 372 | 372 | 372 | 558 | 743 | | Get files/{fileid} | Requests in 1 Minute | 929 | 929 | 929 | 1393 | 1857 | | Get files/{fileid} | Requests in 1 Hour | 1485 | 1485 | 1485 | 2228 | 2970 | | Put files/{fileid} | Requests in 1 Minute | 224 | 224 | 224 | 336 | 447 | | Put files/{fileid} | Requests in 1 Hour | 372 | 372 | 372 | 558 | 743 | | Get files/{fileid}/file | Requests in 1 Minute | 929 | 929 | 929 | 1393 | 1857 | | Get files/{fileid}/file | Requests in 1 Hour | 1485 | 1485 | 1485 | 2228 | 2970 | Note Rate Limits can cause discontinuity in the use case flow. Ensure to check the rate limits before implementing APIs. There are similar limits for "pay as you go" customers. # Asset Management - Async API Specification This is an API that uses Async API specification. You can download the Asset Management Async API specification. [Download Async API Specification](/insights-hub/api_specs/assetmanagement-async-v3-0-0.yaml) # Asset Management Service Async API Service – API Overview ## Available API versions | Region | API version | | -------- | ------------------------------------------------- | | Europe 1 | [3.0.0](api-assetmanagement-async-api-3-0-0.html) | # Asset Management Service Announcement For improved app performance, the following Asset Management API endpoints have a recommended page size. The max page size limit has to be changed to 200 to have a quicker response time and enable parallelism. | Existing behaviour | New behaviour | | ---------------------------------------------------- | --------------------------------------------------- | | GET { { assetmgmturl } } /assets?size=2000 | GET { { assetmgmturl } } / assets?size=200 | | GET { { assettypemgmturl } } / assettypes?size=2000 | GET { { assettypemgmturl } } / assettypes?size=200 | | GET { { assettypemgmturl } } / aspecttypes?size=2000 | GET { { assettypemgmturl } } / aspecttypes?size=200 | ## Idea Independent of your domain, the Asset Management Service supports you in creating digital representations of your physical assets. Such assets could be valves, engines, gas turbines, trains or buildings. The level of detail depends on your use case. ## Access For accessing this service, you need to have the respective roles listed in [Asset Management roles and scopes](https://documentation.mindsphere.io/MindSphere/concepts/concept-roles-scopes.html#asset-management).\ For accessing the Secure Data Sharing (SDS) protected APIs, you need to have appropriate Policy Definitions in place. For the list of supported APIs and required actions, refer to [Required Policy actions by Industrial IoT services](../core-resourceaccessmanagement/api-resourceaccessmanagement-services-actions-map.html#asset-management). ## Basics The API divides the functions into categories, for each category you can use a different controller. This section describes the controllers. ### Aspect Types An Aspect Type is a template for creating multiple aspects with the same variables. Aspects are a data modeling mechanism for assets. Aspects group related data points based on their logical association. An aspect can consist of several variables. You can configure, read or delete your aspect types. An aspect type is accessed by its ID. There are predefined aspect types which are available for all users, but cannot be modified or deleted. ### Asset Types An Asset Type is a pre-configured template for creating multiple assets with the same variables. Assets take on the properties of the type on which they are based. Within the type, you can define which aspects are integrated into the template. You can use a type in various assets and built several interconnections between a type and assets. You can configure, read or delete your types. An asset type is accessed by its ID. Asset Type consists of Variables and Aspects. The Variables defined on an asset type cannot be used for data ingestion so they are called as Static Variables. The Variables and Aspects are also inherited from parent Asset Types. #### Basic Types The Asset Management Service provides basic asset types for users. These are predefined asset types, which are available for all users, but cannot be modified or deleted. The available basic asset types are documented in the [References for Basic Types](api-assetmanagement-references-basictypes.html). #### Default Values The Asset Management Service allows you to assign default values to static (non-time series) variables and static aspect types of an asset type. They are applied following a type-instance concept, so that newly created assets are automatically assigned the default values of their asset type. These values can be overwritten on instance level, if required. Refer to the [Samples for Using Default Values in Asset Types](api-assetmanagement-samples-defaultvalues.html#defining-default-values-for-an-asset-type) to learn more about what you can do with default values. #### Static Variables Asset types consists of Static Variables and Aspects. A variable comprises of properties as `name`, `unit`, `length`, `defaultValue`, `dataType` and `searchable`. Static variables can be updated and deleted. Refer to the [Samples for Updating and Deleting Asset Type Variables](api-assetmanagement-samples-updatingtypes.html) to learn more about updating and deleting variables. Variable properties like `name`, `unit`, `length` and `defaultValue` can be updated however `dataType` and `searchable` cannot be updated. The changed values are propagated to all its Asset instances. Multiple variables can be updated using update variables API, provided the variable names are not interchanged in same request. Updating variables is allowed to the variables defined on an asset type and not the inherited variables. The update operation will update only the given variables and other variables remain as it is. The update request should have `If-Match` header with value of `etag` fetched from the asset type. The response header contains `Etag` for asset type which can be used to update asset type or the variables again. There are certain validations on the update variables operation, in case of failure of a validation no any variable from the request will be updated and the request is failed. On successful update the response code is `204`, in case of failure standard response codes like `400, 401, 403, 404, 412, 500` are returned along with the error message. The update variables API cannot be used to delete variables. Variables can be deleted using asset type update API, after deleting a variable the values overridden on the Asset Instances will also be removed. Multiple variables can be deleted using asset type update API. Deleting variables is allowed only for the variables defined on the asset type and not the inherited variables. ### Assets The following types of assets are distinguished: - Device types: represents a machine, or any object from which data is collected - Agent types: represents the agent (software or physical device) measuring and collecting data of devices, machines, etc. - Hierarchy types: represents the hierarchy levels of an organization - Application types: used by other services to collect data of their applications (e.g., Edge Analytics Application) Using the Asset Management Service, you can configure, read, manage, and assign existing files to assets. Deletion is supported for all assets, but root assets are deleted when their tenant is deleted. #### Hierarchy The Asset Management Service supports hierarchical relationships between assets. Assets inherit their parent's type assignments, but not their assets. They can overwrite the inherited type assignments. The hierarchical path of an asset from the root down via all parent assets is stored in the `hierarchyPath` field. This can be used as navigational aid. An example implementation of hierarchical asset relationships is provided in [Modeling Hierarchical Asset Structures](api-assetmanagement-samples-breadcrumbs.html). #### Structure The structure of an asset can only be read, but not modified. It lists all aspects and variables without their values. You can configure the variables' values by updating the asset. #### Location You can define the location of assets and update or delete the location data. #### Timezone The timezone to be used for timeseries aggregation. By default, it is inherited from the tenant's defaultTimezone, but can be overwritten only during asset creation. The timezone value should be set to a Java time zone ID such as "America/LosAngeles" or "Etc/GMT+2". Time zones that 15 or 45 minutes off a UTC hour are not supported, such as Nepal standard time (UTC+05:45). Time zones that are 30 minutes off a UTC hour are supported, such as India (UTC+05:30). Once an asset is created with a specific timezone, it cannot be changed later. #### Twin Type You can define whether an asset is the digital representation of an actual physical device (performance asset) or it is used for simulation (simulation asset). ### Files You can upload files and assign these files to assets or asset types. Each file can have multiple assignments or none at all. Asset Management File upload API will have validation on matching file content-type vs file name extension to ensure no malicious file is being uploaded. ## Features The Asset Management exposes its API for realizing the following tasks: - Create asset types as templates for assets - Define default values within asset types - Create and manage assets - Model complex asset structures using hierarchy - Upload and assign files to assets ## Limitations - For Private Cloud, `AssetTypes`, `AspectTypes` and variables with same names but different cases should not be created. If same names are used, it may result in inconsistencies in data retrieval. For example: `AspectType` with name `Motor` and `motor` are treated same. ## Example Scenario A brewery has moved a conveyor belt to a different location. A fixed installed camera takes images of the conveyor belt every 2 minutes. The Asset Management API allows you to change the location of the conveyor belt. You can define a new aspect for the conveyor belt like production volume. ## Asset Attribute Calculation Guidelines and Examples ### Asset Attributes ### Asset Modelling Optimization Practices #### New Models ##### Optimize AssetType hierarchy depth Inheritance of AssetType from a parent type results into propagation of variable/attribute definition to child type. ##### Create Purpose Specific AssetType / AspectType Purpose specific AssetType creation reduces the chance of unnecessary variables in Asset instances. ##### Understand the Existing “core” AssetType & AspectTypes and Use the Correct One - A set of core AssetTypes which are made available to provide certain behaviour. - All OOTB AssetTypes have existing definitions which cannot be altered by the end-users. - Therefore, it is recommended to understand the existing type hierarchy and available variable/attribute count associated with them (both direct and inherited) before start using them. - An OOTB AssetType called `core.basicasset` is provided, which does not have any billable attribute. This AssetType can be used as parent AssetType to start inheritance and add purpose specific attributes through customization. - There are some AspectTypes variables defined with core AspectTypes for internal purpose, these variables are also excluded from billing. Note The minimum possible Asset Attribute count for any Asset will be 1 (even if no variables are configured in it). ## Calculation of Asset Attributes and Asset Modeling guidelines Asset attributes per asset type are calculated based on the sum of static variables in the asset type and variables in aspects associated with the asset type. If there are any attributes on the parent asset type, then the asset attributes are inherited by the child asset type. Selected asset types as parent, can have existing aspects and variables that also take part in the calculation, whereas if parent asset types is OOTB then attributes are not inherited to child asset type. The total asset attribute count is computed as the sum of asset attributes for additional asset types. For more details, refer to [Calculation of Asset Attributes](https://documentation.mindsphere.io/MindSphere/apis/advanced-assetmanagement/api-assetmanagement-overview.html#asset-attributes) and [Modeling Recommendation](https://documentation.mindsphere.io/MindSphere/apis/advanced-assetmanagement/api-assetmanagement-overview.html#modeling-recommendation). Disclaimer The Asset Attribute count from the calculator is an estimator and slight variations will be considered.\ When an Asset Instance is created directly from OOTB type, then Asset Attribute count will be increased by 1. ### Existing Models #### Remove unwanted variables/attribute definition from existing AspectType / AssetType - Variable definition on AssetType/AspectType gets directly counted as Asset Attribute through their Asset instances. - Asset Management options are provided to remove existing defined variables from AssetType & AspectType. - Asset Management model operations summary: | Entity | Add | Delete | Update | Updatable | | ---------------------- | --------- | --------- | --------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Asset | Supported | Supported | Supported | - name - description - externalId - parentId - twinType - aspects: add new aspects, remove existing aspects - variables: add new variables, remove existing variables - AssetLocation:country - AssetLocation:region - AssetLocation:locality - AssetLocation:streetAddress - AssetLocation:postalCode - AssetLocation:longitude - AssetLocation:latitude | | AssetType | Supported | Supported | Supported | - name - description - variables \[\]: add more variables, remove existing variables - aspects [] : add more aspects, remove existing aspects | | AssetType: aspects | Supported | Supported | Supported | - name: aspects can be renamed | | AssetType: variables | Supported | Supported | Supported | - name - unit - length: Only for string dataType (can be increased only) - defaultValue | | AspectType | Supported | Supported | Supported | - description - variables [] : add more variables, remove existing variables | | AspectType : variables | Supported | Supported | Supported | - name - unit - length: Only for String dataType (can be increased only) | - Out of all the model operations possible, the following operations can be used to reduce the asset attribute count: | Entity | Operation | Insights Hub APIs | | ---------- | ------------------------------------------------------- | ---------------------------------------------- | | AspectType | Remove unwanted variables | PATCH /aspecttypes/{id}/variables | | AssetType | Remove unwanted variables directly defined on AssetType | PATCH /assettypes/{id} or PUT /assettypes/{id} | | AssetType | Remove unwanted Aspects from AssetType | PATCH /assettypes/{id} or PUT /assettypes/{id} | - For more information about Asset Management API, refer to [Asset Management API specifications](https://documentation.mindsphere.io/MindSphere/apis/advanced-assetmanagement/api-assetmanagement-api.html). ## Modeling Recommendation AssetTypes are template for creating assets. For example, using a single asset type you can create number of assets. AssetType consists of multiple aspects and each aspect is defined with some variables. This association of aspects for a type defines the number of variables present for a type. ### Recommendation for optimum performance - It is recommended not to use more than 100 asset types overall. If 100 types are defined accurately, they are sufficient for creating 1000's of assets. - It is recommended not to use more than 10 variables on a single aspect type. The sum of all the variables comprising of aspect's for a type is the total number of variables belonging to that type. - It is recommended not to use more than 25 aspect types on a single asset type. - It is recommended not to define more than 250 variables on a single asset type. ### Examples for modelling - The type can be modelled with 25 aspects, each consisting of 10 variables, which will result in total 250 variables for the asset type. - The type can be modelled with 10 aspects, each consisting of 10 variables, which will result in total 100 variables for the type. Note This is not a limitation but a recommendation for optimum performance. ### Modeling Recommendations for P&P Tenant Asset Management has imposed technical limits for P&P tenant to safeguard the system and to avoid system exploitation on heavy load exceeding system limits. [API rate limits](api-assetmanagement-apiratelimits.html) for Asset Management are applicable as technical rate limits. The following are the enforced technical limits: | Object Type | Technical limit | | ------------------------------------------------------ | --------------- | | Variable (static and dynamic) defined in an AspectType | 20 | | Aspects associated with an AssetType | 30 | | Static/Direct variable on a AssetType | 25 | | Max depth in AssetType hierarchy | 5 | | Max depth in Asset hierarchy | 20 | | Max Attribute count per asset | 500 | | Max Attribute count per tenant | 500,000 | Note These limits are enforced on Capability Package based environments only. Modelling recommendations (in case of error on exceeded technical limits): | Error Message | Recommendation | | ----------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Aspect type cannot be created as aspect type variables technical limit exceeded for tenant. Please reduce the number of variables in aspect type. | 1. Reduce the number of variables in aspect type. 2. Split variables across multiple aspect types. e.g., 30 variables on an aspect type can be split across 2 aspect types with 15 variables each. | | Asset type cannot be created as asset type aspect technical limit exceeded for tenant. Please reduce the number of assettype-aspecttype associations. | Reduce the number of aspects on asset type by increasing variables per aspect, but make sure that variables per aspect are within limit of 20. | | Asset type cannot be created as asset type direct variable technical limit exceeded for tenant. Please reduce the number of direct variables in asset type. | Reduce the number of direct variables on the asset type by shifting them over to a reusable aspect type. | ## API Page size Recommendation Asset Management suggests using GET APIs in the following ways: - It is recommended to use the GET API with a page size of 200 via concurrent calls for faster data load and optimized performance. For example: `api/assetmanagement/v3/assets?size=200.` - Pagination can be used to get targeted asset model data. Modifying pages dynamically to show assets of interest also contributes to performance improvement. ### Advantages of using GET API with page size 200 - Faster response times when the API is called resulting in a better user experience. - Dynamic page loading with pagination boosts the performance of the Asset Manager. - Easy filtering with compact record sizes keep API transactions light. ### Impact on App development - If the number of assets exceeds 200, multiple calls are required to obtain the complete asset model data. - Below APIs are eligible for page size of 200 - `api/assetmanagement/v3/assets?size=200` - `api/assetmanagement/v3/assettypes?size=200` - `api/assetmanagement/v3/aspecttypes?size=200` ### Transition guide for app development - Get the number of pages that have all the asset data. Example: `api/assetmanagement/v3/assets?page=100000` - Modify the API to start getting assets with a page size of 200. Example: `api/assetmanagement/v3/assets?size=200` - Depending on the number of pages, call the APIs concurrently to get a faster response. - Use pagination to get the desired result instead of skimming through the complete list of assets. ### FAQs 1. Would the requests be throttled after the first 200 records?\ No, in order to get all the records, the API should be called with different page numbers based on the total number of assets. 1. Will there be any data loss after the first 200 records?\ There will not be any data loss after the first 200 records. You can either use pagination to use subsequent records or call the API based on different page numbers to get all the records. 1. What is the impact of this change?\ This API limit has been set to improve the performance of the API. This allows applications to retrieve smaller data sets with better performance than processing all the records. ## Related Links - [IoT TS Aggregates Service](../iot-iottsaggregates/api-iottsaggregates-overview.html) - Create events to document occurrences using [Event Management Service](../advanced-eventmanagement/api-eventmanagement-overview.html) - Notify maintenance personnel in case of an event using [Notification Service](../advanced-notification/api-notification-overview.html) # Asset Management Service – Basic Asset Types This section lists the available basic asset types. They can be recognized by the tenant ID `core`, as in `core.basicdevice`. The following general properties are available on most basic asset types: | Property | Description | | ------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | parentTypeId | The ID of the asset type from which this one was derived | | instantiable | Indicates if instances can be created from this type. Asset types, which are not instantiable are also referred to as abstract types. | | scope | Indicates if an asset type is available to all tenants (public) or only to the tenant it was created in. All basic asset types have a public scope. The scope of asset types created in a tenant is automatically set to `private`. The scope cannot be changed. | For restrictions on the values of individual properties, refer to the [API Specification](api-assetmanagement-api.html). ## Basic Asset The basic asset type is the base/root type of the Asset Management Service. Every asset type is inherited from this type. The basic asset type is an abstract types, so it is not possible to create assets from it. Example ```javascript { "parentTypeId": null, "instantiable": false, "tenantId": "core", "name": "BasicAsset", "description": "Base type for the Asset Management Service.", "scope": "public", "variables": [ { "name": "externalId", "unit": null, "searchable": true, "defaultValue": null, "dataType": "STRING", "length": 255 } ], "aspects": [], "fileAssignments": [], "etag": 1, "_links": { "self": { "href": "{link}" } }, "id": "core.basicasset" } ``` ## Basic Hierarchy Basic hierarchy types are the template for all hierarchy types. They are used for modeling physical, fixed assets, like premises, factories, or equipment to structure assets hierarchically and reduce complexity. The basic hierarchy type is an abstract type, so it is not possible to create assets from it. Example ```javascript { "parentTypeId": "core.basicasset", "instantiable": false, "tenantId": "core", "name": "BasicHierarchy", "description": "Base hierarchy type for the Asset Management Service.", "scope": "public", "variables": [ { "name": "hierarchyModel", "unit": null, "searchable": true, "defaultValue": null, "dataType": "STRING", "length": 255 } ], "aspects": [ { "name": "contactPoint", "aspectType": { "id": "core.contactpoint", "tenantId": "core", "name": "ContactPoint", "category": "static", "scope": "public", "description": "Contact point definition for basic asset hierarchy type.", "variables": [ { "name": "telephoneNumber", "unit": null, "searchable": true, "qualityCode": false, "defaultValue": null, "dataType": "STRING", "length": 255 }, { "name": "faxNumber", "unit": null, "searchable": true, "qualityCode": false, "defaultValue": null, "dataType": "STRING", "length": 255 }, { "name": "email", "unit": null, "searchable": true, "qualityCode": false, "defaultValue": null, "dataType": "STRING", "length": 255 }, { "name": "contactType", "unit": null, "searchable": true, "qualityCode": false, "defaultValue": null, "dataType": "STRING", "length": 255 } ], "etag": {etagValue}, "_links": { "self": { "href": "{link}" } } } } ], "fileAssignments": [], "etag": {etagValue}, "_links": { "self": { "href": "{link}" }, "parent": { "href": "{link}" } }, "id": "core.basichierarchy" } ``` ### Root Hierarchy The root hierarchy type is the template for root types. These are used as the starting point of hierarchy models of a tenant's or subtenant's structure. When a tenant or subtenant is created, a root asset has to be created. All root types must be inherited from the root hierarchy type. The root hierarchy type is an abstract type, so it is not possible to create assets from it. Users cannot create assets derived from a root hierarchy type via Asset Management REST calls. Example ```javascript { "parentTypeId": "core.basichierarchy", "instantiable": false, "tenantId": "core", "name": "RootHierarchy", "description": "Root type for asset hierarchy.", "scope": "public", "variables": [], "aspects": [], "fileAssignments": [], "etag": {etagValue}, "_links": { "self": { "href": "{link}" }, "parent": { "href": "{link}" } }, "id": "core.roothierarchy" } ``` #### Basic Enterprise This is the template, from which a tenant's root asset must be created. Example ```javascript { "parentTypeId": "core.roothierarchy", "instantiable": true, "tenantId": "core", "name": "BasicEnterprise", "description": "Basic Enterprise type for the Asset Management Service", "scope": "public", "variables": [ { "name": "sameAs", "unit": null, "searchable": true, "defaultValue": null, "dataType": "STRING", "length": 255 }, { "name": "legalName", "unit": null, "searchable": true, "defaultValue": null, "dataType": "STRING", "length": 255 }, { "name": "organizationType", "unit": null, "searchable": true, "defaultValue": null, "dataType": "STRING", "length": 255 }, { "name": "url", "unit": null, "searchable": true, "defaultValue": null, "dataType": "STRING", "length": 255 } ], "aspects": [], "fileAssignments": [], "etag": {etagValue}, "_links": { "self": { "href": "{link}" }, "parent": { "href": "{link}" } }, "id": "core.basicenterprise" } ``` #### Basic Subtenant This is the template, from which a subtenant's root asset must be created inside an existing tenant. Example ```javascript { "parentTypeId": "core.roothierarchy", "instantiable": true, "tenantId": "core", "name": "BasicSubtenant", "description": "SubTenant type for the Asset Management Service", "scope": "public", "variables": [ { "name": "displayName", "unit": null, "searchable": true, "defaultValue": null, "dataType": "STRING", "length": 255 } ], "aspects": [], "fileAssignments": [], "etag": {etagValue}, "_links": { "self": { "href": "{link}" }, "parent": { "href": "{link}" } }, "id": "core.basicsubtenant" } ``` ### Basic Site The basic site type is used as a template for a site like a factory, office, etc. The [Modeling Assets Section](api-assetmanagement-samples-modelingassets.html) illustrates how to use the basic site type. Example ```javascript { "parentTypeId": "core.basichierarchy", "instantiable": true, "tenantId": "core", "name": "BasicSite", "description": "Site type for creating asset hierarchy levels.", "scope": "public", "variables": [], "aspects": [], "fileAssignments": [], "etag": {etagValue}, "_links": { "self": { "href": "{link}" }, "parent": { "href": "{link}" } }, "id": "core.basicsite" } ``` ### Basic Area The basic area is used as a template for an area inside a site, like a bottling area in a beer factory or a floor in an office building. Example ```javascript { "parentTypeId": "core.basichierarchy", "instantiable": true, "tenantId": "core", "name": "BasicArea", "description": "Area type for creating asset hierarchy levels.", "scope": "public", "variables": [], "aspects": [], "fileAssignments": [], "etag": {etagValue}, "_links": { "self": { "href": "{link}" }, "parent": { "href": "{link}" } } "id": "core.basicarea" } ``` ## Basic Agent The basic agent type is the template for the different types of boxes or agents. The basic agent type is an abstract type, so it is not possible to create assets from it. Example ```javascript { "parentTypeId": "core.basicasset", "instantiable": false, "tenantId": "core", "name": "BasicAgent", "description": "Basic agent type for the Asset Management Service.", "scope": "public", "variables": [], "aspects": [ { "name": "status", "aspectType": { "id": "core.agentstatus", "tenantId": "core", "name": "AgentOnlineStatus", "category": "dynamic", "scope": "public", "description": "Online status of an agent typed asset.", "variables": [ { "name": "onlineStatus", "unit": null, "searchable": false, "qualityCode": false, "defaultValue": null, "dataType": "BOOLEAN", "length": null } ], "etag": {etagValue}, "_links": { "self": { "href": "{link}" } } } } ], "fileAssignments": [], "etag": {etagValue}, "_links": { "self": { "href": "{link}" }, "parent": { "href": "{link}" } } "id": "core.basicagent" } ``` ### MindConnect Nano This asset type is the template for MindConnect Nano connectivity elements. Example ```javascript { "parentTypeId": "core.basicagent", "instantiable": true, "tenantId": "core", "name": "MindConnectNano", "description": "MindConnect Nano Agent asset type", "scope": "public", "variables": [], "aspects": [], "fileAssignments": [], "etag": {etagValue}, "_links": { "self": { "href": "{link}" }, "parent": { "href": "{link}" } }, "id": "core.mcnano" } ``` ### MindConnect Lib This asset type is the template for MindConnect Library connectivity elements. Example ```javascript { "parentTypeId": "core.basicagent", "instantiable": true, "tenantId": "core", "name": "MindConnectLib", "description": "MindConnect Lib Agent asset type", "scope": "public", "variables": [], "aspects": [], "fileAssignments": [], "etag": {etagValue}, "_links": { "self": { "href": "{link}" }, "parent": { "href": "{link}" } }, "id": "core.mclib" } ``` ### MindConnect IoT 2040 This asset type is the template for MindConnect IoT 2040 connectivity elements. Example ```javascript { "parentTypeId": "core.basicagent", "instantiable": true, "tenantId": "core", "name": "MindConnectIoT2040", "description": "MindConnect IoT 2040 Agent asset type", "scope": "public", "variables": [], "aspects": [], "fileAssignments": [], "etag": {etagValue}, "_links": { "self": { "href": "{link}" }, "parent": { "href": "{link}" } }, "id": "core.mciot2040" } ``` ### MindConnect FB1500 Note This basic type is deprecated. If you want to connect and represent the S7-1500, use the MindConnect Library. Example ```javascript { "parentTypeId": "core.basicagent", "instantiable": true, "tenantId": "core", "name": "MindConnectFB1500", "description": "MindConnect FB 1500 Agent asset type", "scope": "public", "variables": [], "aspects": [], "fileAssignments": [], "etag": {etagValue}, "_links": { "self": { "href": "{link}" }, "parent": { "href": "{link}" } }, "id": "core.mcfb1500" } ``` ### MindConnect Integration Example ```javascript { "parentTypeId": "core.basicagent", "instantiable": true, "tenantId": "core", "name": "MindConnectIntegration", "description": "MindConnect Integration Agent asset type", "scope": "public", "variables": [], "aspects": [], "fileAssignments": [], "etag": {etagValue}, "_links": { "self": { "href": "{link}" }, "parent": { "href": "{link}" } }, "id": "core.mcintegration" } ``` ### Industrial Edge Example ```javascript { "parentTypeId": "core.basicagent", "instantiable": true, "tenantId": "core", "name": "IndustrialEdge", "description": "Industrial Edge Agent asset type", "scope": "public", "variables": [], "aspects": [], "fileAssignments": [], "etag": {etagValue}, "_links": { "self": { "href": "{link}" }, "parent": { "href": "{link}" } }, "id": "core.industrialEdge" } ``` ## Basic Device The basic device type is the template for machines or other assets to collect data from. Every device asset type is inherited from the basic device type. The basic device type is an abstract asset type, so it is not possible to create assets from it. Example ```javascript { "parentTypeId": "core.basicasset", "instantiable": false, "tenantId": "core", "name": "BasicDevice", "description": "Basic device asset type for the Asset Management Service.", "scope": "public", "variables": [ { "name": "manufacturer", "unit": null, "searchable": true, "defaultValue": null, "dataType": "STRING", "length": 255 } ], "aspects": [ { "name": "status", "aspectType": { "id": "core.assetstatus", "tenantId": "core", "name": "DeviceAssetStatus", "category": "static", "scope": "public", "description": "Status of a device typed asset.", "variables": [ { "name": "connection_timestamp", "unit": null, "searchable": true, "qualityCode": false, "defaultValue": null, "dataType": "TIMESTAMP", "length": null }, { "name": "connection", "unit": null, "searchable": true, "qualityCode": false, "defaultValue": null, "dataType": "INT", "length": null }, { "name": "health_unconfirmed", "unit": null, "searchable": true, "qualityCode": false, "defaultValue": null, "dataType": "INT", "length": null }, { "name": "health_timestamp", "unit": null, "searchable": true, "qualityCode": false, "defaultValue": null, "dataType": "TIMESTAMP", "length": null }, { "name": "health", "unit": null, "searchable": true, "qualityCode": false, "defaultValue": null, "dataType": "INT", "length": null }, { "name": "health_unconfirmed_timestamp", "unit": null, "searchable": true, "qualityCode": false, "defaultValue": null, "dataType": "TIMESTAMP", "length": null } ], "etag": {etagValue}, "_links": { "self": { "href": "{link}" } } } } ], "fileAssignments": [], "etag": {etagValue}, "_links": { "self": { "href": "{link}" }, "parent": { "href": "{link}" } }, "id": "core.basicdevice" } ``` ### Basic Sinumerik Asset The basic Sinumeric asset type is the template for Sinumerik devices. The basic Sinumerik asset type is an abstract asset type, so it is not possible to create assets from it. Example ```javascript { "parentTypeId": "core.basicdevice", "instantiable": false, "tenantId": "core", "name": "BasicSinumerikAsset", "description": "Basic asset type for Sinumerik devices.", "scope": "public", "variables": [], "aspects": [ { "name": "CH1_BasicConfig", "aspectType": { "id": "core.sinumerikbasicconfig", "tenantId": "core", "name": "SinumerikBasicConfig", "category": "dynamic", "scope": "public", "description": "CH1_BasicConfig for Sinumerik typed asset.", "variables": [ { "name": "NCProgramStatus", "unit": null, "searchable": true, "qualityCode": false, "defaultValue": null, "dataType": "DOUBLE", "length": null }, { "name": "Spindleoverride", "unit": "%", "searchable": true, "qualityCode": false, "defaultValue": null, "dataType": "DOUBLE", "length": null }, { "name": "StopCond", "unit": null, "searchable": true, "qualityCode": false, "defaultValue": null, "dataType": "DOUBLE", "length": null }, { "name": "ProtectionLevel", "unit": null, "searchable": true, "qualityCode": false, "defaultValue": null, "dataType": "DOUBLE", "length": null }, { "name": "NCProgram", "unit": null, "searchable": true, "qualityCode": false, "defaultValue": null, "dataType": "STRING", "length": 255 }, { "name": "Opmode", "unit": null, "searchable": true, "qualityCode": false, "defaultValue": null, "dataType": "DOUBLE", "length": null }, { "name": "NrOfAlarms", "unit": null, "searchable": true, "qualityCode": false, "defaultValue": null, "dataType": "DOUBLE", "length": null }, { "name": "Feedoverride", "unit": "%", "searchable": true, "qualityCode": false, "defaultValue": null, "dataType": "DOUBLE", "length": null } ], "etag": {etagValue}, "_links": { "self": { "href": "{link}" } } } }, { "name": "Alarms", "aspectType": { "id": "core.sinumerikbasicalarms", "tenantId": "core", "name": "SinumerikBasicAlarms", "category": "dynamic", "scope": "public", "description": "Alarms for Sinumerik typed asset.", "variables": [ { "name": "CurrentAlarms", "unit": null, "searchable": true, "qualityCode": false, "defaultValue": null, "dataType": "BIG_STRING", "length": 100000 } ], "etag": {etagValue}, "_links": { "self": { "href": "{link}" } } } }, { "name": "MachineModel", "aspectType": { "id": "core.sinumerikbasicmachinemodel", "tenantId": "core", "name": "SinumerikBasicMachineModel", "category": "dynamic", "scope": "public", "description": "Machinemodel for Sinumerik typed asset.", "variables": [ { "name": "Data", "unit": null, "searchable": true, "qualityCode": false, "defaultValue": null, "dataType": "BIG_STRING", "length": 100000 } ], "etag": {etagValue}, "_links": { "self": { "href": "{link}" } } } }, { "name": "CH1_MachineStatus", "aspectType": { "id": "core.sinumerikbasicmachinestatus", "tenantId": "core", "name": "SinumerikBasicMachineStatus", "category": "dynamic", "scope": "public", "description": "CH1_MachineStatus for Sinumerik typed asset.", "variables": [ { "name": "MachineStatus", "unit": "%", "searchable": true, "qualityCode": false, "defaultValue": null, "dataType": "DOUBLE", "length": null } ], "etag": {etagValue}, "_links": { "self": { "href": "{link}" } } } }, { "name": "Startup", "aspectType": { "id": "core.sinumerikbasicstartup", "tenantId": "core", "name": "SinumerikBasicStartup", "category": "dynamic", "scope": "public", "description": "Startup for Sinumerik typed asset.", "variables": [ { "name": "shutdownTime", "unit": null, "searchable": true, "qualityCode": false, "defaultValue": null, "dataType": "STRING", "length": 255 }, { "name": "bootTime", "unit": null, "searchable": true, "qualityCode": false, "defaultValue": null, "dataType": "STRING", "length": 255 } ], "etag": {etagValue}, "_links": { "self": { "href": "{link}" } } } }, { "name": "AgentOnlineStatus", "aspectType": { "id": "core.agentstatus", "tenantId": "core", "name": "AgentOnlineStatus", "category": "dynamic", "scope": "public", "description": "Online status of an agent typed asset.", "variables": [ { "name": "onlineStatus", "unit": null, "searchable": false, "qualityCode": false, "defaultValue": null, "dataType": "BOOLEAN", "length": null } ], "etag": {etagValue}, "_links": { "self": { "href": "{link}" } } } } ], "fileAssignments": [], "etag": {etagValue}, "_links": { "self": { "href": "{link}" }, "parent": { "href": "{link}" } }, "id": "core.basicsinumerikasset" } ``` ## Basic Application The basic application type is the template for applications, which ingest data into Insights Hub. The basic application type is an abstract asset type, so it is not possible to create assets from it. Example ```javascript { "parentTypeId": "core.basicasset", "instantiable": false, "tenantId": "core", "name": "BasicApplication", "description": "Basic asset type for Applications", "scope": "public", "variables": [], "aspects": [], "fileAssignments": [], "etag": {etagValue}, "_links": { "self": { "href": "{link}" }, "parent": { "href": "{link}" } }, "id": "core.basicapplication" } ``` ### Edge Basic Application This asset type is the template for all Edge applications. The Edge basic application type is an abstract asset type, so it is not possible to create assets from it. Example ```javascript { "parentTypeId": "core.basicapplication", "instantiable": false, "tenantId": "core", "name": "EdgeBasicApplication", "description": "Represents the root asset type for all edge applications", "scope": "public", "variables": [], "aspects": [ { "name": "status", "aspectType": { "id": "core.assetstatus", "tenantId": "core", "name": "DeviceAssetStatus", "category": "static", "scope": "public", "description": "Status of a device typed asset.", "variables": [ { "name": "connection_timestamp", "unit": null, "searchable": true, "qualityCode": false, "defaultValue": null, "dataType": "TIMESTAMP", "length": null }, { "name": "connection", "unit": null, "searchable": true, "qualityCode": false, "defaultValue": null, "dataType": "INT", "length": null }, { "name": "health_unconfirmed", "unit": null, "searchable": true, "qualityCode": false, "defaultValue": null, "dataType": "INT", "length": null }, { "name": "health_timestamp", "unit": null, "searchable": true, "qualityCode": false, "defaultValue": null, "dataType": "TIMESTAMP", "length": null }, { "name": "health", "unit": null, "searchable": true, "qualityCode": false, "defaultValue": null, "dataType": "INT", "length": null }, { "name": "health_unconfirmed_timestamp", "unit": null, "searchable": true, "qualityCode": false, "defaultValue": null, "dataType": "TIMESTAMP", "length": null } ], "etag": {etagValue}, "_links": { "self": { "href": "{link}" } } } } ], "fileAssignments": [], "etag": {etagValue}, "_links": { "self": { "href": "{link}" }, "parent": { "href": "{link}" } }, "id": "core.edgebasicapplication" } ``` #### Edge Analytics Application This asset type is the template for the Edge Analytics Application present on the Edge device. Example ```javascript { "parentTypeId": "core.edgebasicapplication", "instantiable": true, "tenantId": "core", "name": "EdgeAnalyticsApplication", "description": "Represents the Edge Analytics Application present on the Edge device", "scope": "public", "variables": [], "aspects": [ { "name": "EdgeAnalyticsApplicationStatus", "aspectType": { "id": "core.edgeanalyticsapplicationstatus", "tenantId": "core", "name": "EdgeAnalyticsApplicationStatus", "category": "dynamic", "scope": "public", "description": "Status of the Edge Analytics Application", "variables": [ { "name": "CPU_usage", "unit": "%", "searchable": false, "qualityCode": false, "defaultValue": null, "dataType": "INT", "length": null }, { "name": "Project_state", "unit": null, "searchable": false, "qualityCode": false, "defaultValue": null, "dataType": "BOOLEAN", "length": null }, { "name": "Server_state", "unit": null, "searchable": false, "qualityCode": false, "defaultValue": null, "dataType": "BOOLEAN", "length": null } ], "etag": {etagValue}, "_links": { "self": { "href": "{link}" } } } } ], "fileAssignments": [], "etag": {etagValue}, "_links": { "self": { "href": "{link}" }, "parent": { "href": "{link}" } }, "id": "core.edgeanalyticsapplication" } ``` #### Open Edge Device This asset type is the template for the Open Edge device. Example ```javascript { "parentTypeId": "core.basicasset", "instantiable": false, "tenantId": "core", "name": "EdgeDevice", "description": "Open Edge device asset type for Asset Management Service.", "scope": "public", "variables": [], "aspects": [], "fileAssignments": [], "sharing": { "modes": [] }, "etag": 0, "_links": { "self": { "href": "https://gateway.eu1-int.mindsphere.io/api/assetmanagement/v3/assettypes/core.edgedevice" }, "parent": { "href": "https://gateway.eu1-int.mindsphere.io/api/assetmanagement/v3/assettypes/core.basicasset" } }, "id": "core.edgedevice" } ``` # Asset Management Service – Filtering All endpoints with the parameter `filter` support querying for filterable properties. Queries for other properties (e.g. variables, aspects) are not supported. When reading assets, all assets are filtered for main tenant and subtenant. In addition, you can supply one or more of the following field names in the filter queries (all other field names will be ignored): - `assetId` - `tenantId` - `name` - `externalId` - `t2tenant` - `subTenant` - `timezone` - `twinType` - `deleted` - `parentId` - `typeId` - `fileId` - `hasType` (see also [Special fields](#field-hastype)) Wildcards are not supported. Timestamps must be in the following format: `2018-01-01T00:00:00.00Z` When a filter has no matches, the response will be empty. ## Filter functions ### Function `eq` (equals) ```json { "name": "MyMotor" } ``` or ```json { "name": { "eq": "MyMotor" } } ``` ### Function `in` The function `in` is very similar to `equals`, but instead matching to one value, it checks an array of values, and matches for any of them. ```json { "name": { "in": { "value": ["MyMotor", "MyEngine"] } } } ``` ### Function `endsWith` ```json { "name": { "endsWith": "Motor" } } ``` ### Function `startsWith` ```json { "name": { "startsWith": "My" } } ``` ### Function `contains` ```json { "name": { "contains": "yMot" } } ``` ### Function `before` ```json { "deleted": { "before": "2018-01-01T00:00:00.00Z" } } ``` ### Function `after` ```json { "deleted": { "after": "2018-01-01T00:00:00.00Z" } } ``` ### Function `between` ```json { "deleted": { "between": "[2018-01-01T00:00:00.00Z, 2018-01-31T00:00:00.00Z)" } } ``` ## Filter operations ### Operation `not` ```json { "not": { "name": { "startsWith": "My" } } } ``` or ```json { "not": { "name": "MyMotor" } } ``` ### Operation `or` ```json { "name": { "or": [{ "eq": "MyMotor" }, { "endsWith": "Motor" }] } } ``` or ```json { "or": { "deleted": { "eq": null }, "name": { "startsWith": "My" } } } ``` ### Operation `and` ```json { "name": "MyMotor", "deleted": null } ``` or ```json { "and": { "deleted": { "eq": null }, "name": { "startsWith": "My" } } } ``` ## Special fields ### Field `hasType` Using `hasType` you can filter for any asset type and its descendants. E.g., all assets of type `core.motor` or of any type which is derived from the asset type `core.motor`. ```json { "hasType": "core.motor" } ``` Note While `hasType` matches for any asset type in the type hierarchy, the field `typeId` only matches with the given asset's typeId and thus does not return descendants. # Asset Management Service – Pagination When working with a tenant, it typically collects more and more assets, asset types, aspect types and files. When exploring these using the respective `GET` endpoints, the responses would contain object lists with several hundreds of entries. In order to break the results down into processable chunks, the Asset Management Service provides a pagination functionality. This functionality divides the results into pages, which only hold a defined number of results. The following endpoints support the pagination functionality: ```http GET /assettype ``` ```http GET /aspecttype ``` ```http GET /assets ``` ```http GET /files ``` ```http GET /assets/{assetId}/variables ``` ```http GET /assets/{assetId}/aspects ``` ## Pagination Parameters | Parameter | Description | Default Value | | --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------- | | `size` | Defines how many objects are returned per page. (optional) | 10 | | `page` | Each page only holds as many objects as defined by the `size` parameter and by default, only the first page is returned. Use this parameter to change the page number to be returned. (optional) | 0 | | `sort` | Defines based on which object attributes the results are sorted. (optional) | name in ascending order | As the table indicates, the pagination parameters are optional. By default, the endpoints return the first 10 elements in alphabetical order. The end of the response lists the value of the `size` parameter, the total number of elements, the total number of pages, and the page number of the returned page, for example: ```javascript "page": { "size": 10, "totalElements": 433, "totalPages": 44, "number": 0 } ``` ### Size The number of objects returned per page is set using the `size` parameter. Its default value is 10. Note that changing the `size` value affects the total number of pages, as the total number of objects is divided into bigger or smaller chunks. It is recommended to provide the page size of 200 for asset management GET APIs: GET /assettype, GET /aspecttype and GET /assets. ### Page By default, the first page of a result is returned. Its page number is 0, as is the default value of the `page` parameter. If a result consists of multiple pages, select the page to be returned using this parameter. ### Sorting Using the `sort` parameter, you can modify both the sorting attributes and the order of the sorting. The input for this parameter is case-sensitive and accepts the name of the object attribute to base the sorting on. All object attributes specified as filterable are allowed as input.\ For sorting in descending order, append the name of object attribute with `,desc`. This requires that the object attribute is set explicitly, even if you want to use the default value. # Asset Management Service – Modeling Hierarchical Asset Structures The following example illustrates how to model hierarchical structures of assets. In this example, a simple robotic arm inside a car factory is created, which is equipped with an IoT 2040 box. The final model will consist of 5 levels of hierarchy: 1. The root asset representing the tenant 1. A site asset representing the factory 1. An area asset representing the assembly area 1. A device asset representing a simple robotic arm 1. An agent asset representing an IoT 2040 box ## Creating the Site Asset The factory asset is created using the following request: ```http POST /assets ``` The following JSON structure is given to define the site asset: ```javascript { "name": "MindCar", "externalId": "SN 123456-123-123456", "description": "Factory of MindCar", "location": { "country": "Austria", "region": "Tyrol", "locality": "Innsbruck", "streetAddress": "Industriestraße 21 A/II", "postalCode": "6020", "longitude": 53.5125546, "latitude": 9.9763411 }, "typeId": "core.basicsite", "parentId": "{assetId}", "timezone": "Europe/Berlin" } ``` The response of the call contains the newly created asset instance, so you can validate the correct creation: Response ```javascript { "assetId": "{assetId}", "tenantId": "{tenantName}", "name": "MindCar", "etag": {etagValue}, "externalId": "SN 123456-123-123456", "t2Tenant": null, "subTenant": null, "description": "Factory of MindCar", "timezone": "Europe/Berlin", "parentId": "{parentAssetId}", "typeId": "core.basicsite", "location": { "country": "Austria", "region": "Tyrol", "locality": "Innsbruck", "streetAddress": "Industriestraße 21 A/II", "postalCode": "6020", "longitude": 53.5125546, "latitude": 9.9763411 }, "fileAssignments": [], "variables": [], "aspects": [], "locks": [], "hierarchyPath": [ { "assetId": "{rootAssetId}", "name": "{tenantName}" } ], "deleted": null, "_links": { "self": { "href": "{link}" }, "aspects": { "href": "{link}" }, "variables": { "href": "{link}" }, "location": { "href": "{link}" }, "parent": { "href": "{link}" } } } ``` Lines 26 to 31 contain the hierarchy path up to the root asset, including the parent asset's ID and its name: ```javascript "hierarchyPath": [ { "assetId": "{rootAssetId}", "name": "{tenantName}" } ] ``` Note The returned hierarchy path may differ: Tenant users receive the full path to the tenant's root asset, whereas subtenant users only receive the path to the subtenant's root asset. ## Creating the Area Asset The area asset is created using the same request as before, but using the following JSON structure to define the area asset: ```javascript { "name": "AssemblyArea", "externalId": "SN 123456-123-123456", "description": "Assembly area of MindCar", "location": { "country": "Austria", "region": "Tyrol", "locality": "Innsbruck", "streetAddress": "Industriestraße 21 A/II", "postalCode": "6020", "longitude": 53.5125546, "latitude": 9.9763411 }, "typeId": "core.basicarea", "parentId": "{assetId}", "timezone": "Europe/Berlin" } ``` The response of the call contains the newly created asset instance, so you can validate the correct creation: Response ```javascript { "assetId": "{assetId}", "tenantId": "{tenantName}", "name": "AssemblyArea", "etag": {etagValue}, "externalId": "SN 123456-123-123456", "t2Tenant": null, "subTenant": null, "description": "Assembly area of MindCar", "timezone": "Europe/Berlin", "parentId": "{parentId}", "typeId": "core.basicarea", "location": { "country": "Austria", "region": "Tyrol", "locality": "Innsbruck", "streetAddress": "Industriestraße 21 A/II", "postalCode": "6020", "longitude": 53.5125546, "latitude": 9.9763411 }, "fileAssignments": [], "variables": [], "aspects": [], "locks": [], "hierarchyPath": [ { "assetId": "{rootAssetId}", "name": "{tenantName}" }, { "assetId": "{siteAssetId}", "name": "MindCar" } ], "deleted": null, "_links": { "self": { "href": "{link}" }, "aspects": { "href": "{link}" }, "variables": { "href": "{link}" }, "location": { "href": "{link}" }, "parent": { "href": "{link}" } } } ``` Lines 26 to 35 contain the hierarchy path up to the root asset, which consists of two levels in this case. The hierarchy path always starts at the root asset and ends with the direct parent asset: ```javascript "hierarchyPath": [ { "assetId": "{rootAssetId}", "name": "{tenantName}" }, { "assetId": "{siteAssetId}", "name": "MindCar" } ] ``` ## Creating the Device Asset When creating a new asset, the required aspect types and asset type must be defined first. In this example, the device is a simple robotic arm and the variable to be measured is its rotation. ### Creating the Aspect Type The aspect type for the rotation variable is created using the following request: ```http PUT /aspecttype/{id} ``` The following input is given for the `id` parameter: ```json {tenantName}.rotation ``` The following JSON structure is given to define the aspect type: ```json { "name": "rotation", "category": "dynamic", "scope": "private", "description": "Rotation of the robotic arm", "variables": [ { "name": "rotation", "dataType": "DOUBLE", "unit": "degree", "searchable": true } ] } ``` The response of the call contains the newly created aspect type, so you can validate the correct creation: ```javascript { "id": "{tenantName}.rotation", "tenantId": "{tenantName}", "name": "rotation", "category": "dynamic", "scope": "private", "description": "Rotation of the robotic arm", "variables": [ { "name": "rotation", "unit": "degree", "searchable": true, "qualityCode": false, "dataType": "DOUBLE", "defaultValue": null, "length": null } ], "etag": {etagValue}, "_links": { "self": { "href": "{link}" } } } ``` ### Creating the Asset Type After creating the required aspect type, the asset type for the robotic arm can be created using the following request: ```http PUT /assecttype/{id} ``` The following input is given for the `id` parameter: ```json {tenantName}.roboticarm ``` The following JSON structure is given to define the asset type: ```javascript { "name": "roboticArm", "description": "Robotic arm assembling cars", "parentTypeId": "core.basicdevice", "instantiable": true, "scope": "private", "aspects": [ { "name": "rotation", "aspectTypeId": "{tenantName}.rotation" } ] } ``` The response of the call contains the newly created asset type, so you can validate the correct creation: Response ```javascript { "parentTypeId": "core.basicdevice", "instantiable": true, "tenantId": "{tenantName}", "name": "roboticArm", "description": "Robotic arm assembling cars", "scope": "private", "variables": [], "aspects": [ { "name": "rotation", "aspectType": { "id": "{tenantName}.rotation", "tenantId": "{tenantName}", "name": "rotation", "category": "dynamic", "scope": "private", "description": "Rotation of the robotic arm", "variables": [ { "name": "rotation", "unit": "degree", "searchable": true, "qualityCode": false, "defaultValue": null, "dataType": "DOUBLE", "length": null } ], "etag": {etagValue}, "_links": { "self": { "href": "{link}" } } } } ], "fileAssignments": [], "etag": {etagValue}, "_links": { "self": { "href": "{link}" }, "parent": { "href": "{link}" } }, "id": "{tenantName}.roboticarm" } ``` ### Creating the Asset After creating the required asset type, the device asset for the robotic arm can be created using the following request: ```http POST /assets ``` The following JSON structure is given to define the device asset: ```javascript { "name": "roboticarm", "externalId": "SN 123456-123-123456", "description": "Robotic arm assembling cars", "typeId": "{tenantName}.roboticarm", "parentId": "{assetId}", "timezone": "Europe/Berlin" } ``` The response of the call contains the newly created asset instance, so you can validate the correct creation: Response ```javascript { "assetId": "{assetId}", "tenantId": "{tenantName}", "name": "roboticarm", "etag": {etagValue}, "externalId": "SN 123456-123-123456", "t2Tenant": null, "subTenant": null, "description": "Robotic arm assembling cars", "timezone": "Europe/Berlin", "parentId": "{parentId}", "typeId": "{tenantName}.roboticarm", "location": { "country": "Austria", "region": "Tyrol", "locality": "Innsbruck", "streetAddress": "Industriestraße 21 A/II", "postalCode": "6020", "longitude": 53.5125546, "latitude": 9.9763411 }, "fileAssignments": [], "variables": [], "aspects": [], "locks": [], "hierarchyPath": [ { "assetId": "{rootAssetId}", "name": "{tenantName}" }, { "assetId": "{assetId}", "name": "MindCar" }, { "assetId": "{assetId}", "name": "AssemblyArea" } ], "deleted": null, "_links": { "self": { "href": "{link}" }, "aspects": { "href": "{link}" }, "variables": { "href": "{link}" }, "location": { "href": "{link}" }, "parent": { "href": "{link}" } } } ``` Lines 26 to 35 contain the hierarchy path up to the root asset. The hierarchy path shows all the parents up to the root asset with their `assetId` and `name`, starting from the root and finishing at the direct parent the area. ```javascript "hierarchyPath": [ { "assetId": "{rootAssetId}", "name": "{tenantName}" }, { "assetId": "{assetId}", "name": "MindCar" }, { "assetId": "{assetId}", "name": "AssemblyArea" } ] ``` ## Creating the IoT 2040 Agent Asset Finally, an IoT 2040 agent asset is created using the following request: ```http POST /assets ``` The following JSON structure is given to define the agent asset: ```javascript { "name": "IoT2040", "externalId": "SN 123456-123-123456", "description": "IoT2040", "typeId": "core.mciot2040", "parentId": "{assetId}", "timezone": "Europe/Berlin" } ``` The response of the call contains the newly created asset instance, so you can validate the correct creation: Response ```javascript { "assetId": "{assetId}", "tenantId": "{tenantValue}", "name": "IoT2040", "etag": {etagValue}, "externalId": "SN 123456-123-123456", "t2Tenant": null, "subTenant": null, "description": "IoT2040", "timezone": "Europe/Berlin", "parentId": "{assetId}", "typeId": "core.mciot2040", "location": { "country": "Austria", "region": "Tyrol", "locality": "Innsbruck", "streetAddress": "Industriestraße 21 A/II", "postalCode": "6020", "longitude": 53.5125546, "latitude": 9.9763411 }, "fileAssignments": [], "variables": [], "aspects": [], "locks": [], "hierarchyPath": [ { "assetId": "{rootAssetId}", "name": "{tenantName}" }, { "assetId": "{assetId}", "name": "MindCar" }, { "assetId": "{assetId}", "name": "AssemblyArea" }, { "assetId": "{assetId}", "name": "roboticarm" } ], "deleted": null, "_links": { "self": { "href": "{link}" }, "aspects": { "href": "{link}" }, "variables": { "href": "{link}" }, "location": { "href": "{link}" }, "parent": { "href": "{link}" } } } ``` The hierarchy path shows all the parents up to the root asset with their assetId and name starting from the root and finishing at the direct parent the robotic arm. ```javascript "hierarchyPath": [ { "assetId": "{rootAssetId}", "name": "{tenantName}" }, { "assetId": "{assetId}", "name": "MindCar" }, { "assetId": "{assetId}", "name": "AssemblyArea" }, { "assetId": "{assetId}", "name": "roboticarm" } ] ``` # Asset Management Service – Using Default Values in Asset Types The following example illustrates how to create an asset type and define default values for one of its properties using the Asset Management Service. Here, an asset type `Valve`, which has a property named `color`, is created. A new valve shall be gray by default, but as this example shows, the default value can be overwritten, if required. Note {tenantName} used in below examples is the name of tenant, can be obtained using TenantManagements /tenantInfo API [TenantManagement](https://documentation.mindsphere.io/MindSphere/apis-xcelerator/core-tenantmanagement/api-tenantmanagement-api.html). ## Defining Default Values for an Asset Type Create the asset type using the following request - replace `{tenantId}` by your tenant's ID: ```http PUT /assettypes/{tenantName}.Valve ``` Include the following JSON structure in the payload to define the asset type and its default value for the `color` property: ```json { "name": "Valve", "description": "General purpose valve", "parentTypeId": "core.basicdevice", "instantiable": true, "scope": "private", "variables": [ { "name": "color", "dataType": "STRING", "searchable": true, "length": 10, "defaultValue": "gray" } ] } ``` Create an asset instance of the newly created asset type using the following request: ```http POST /assets/ ``` Include the following JSON structure in the payload to define the asset - replace `{tenantID}` by your tenant's ID and `{parentAssetId}` by the [ID of your preferred parent asset](#finding-out-the-parentid-of-an-asset): ```json { "name": "Valve-001", "externalId": "SN 123456-123-123456", "description": "Valve 001 installed somewhere", "typeId": "{tenantName}.Valve", "parentId": "{parentAssetId}", "timezone": "Europe/Berlin" } ``` The response of the call contains the newly created asset instance so you can validate the correct creation: ```json { "assetId": "{assetId}", "tenantId": "{tenantId}", "name": "Valve-001", "etag": "{etagValue}", "externalId": "SN 123456-123-123456", "t2Tenant": null, "subTenant": null, "description": "Valve 001 installed somewhere", "timezone": "Europe/Berlin", "parentId": "{parentAssetId}", "typeId": "{tenantName}.Valve", "location": null, "fileAssignments": [], "variables": [ { "name": "color", "value": "gray" } ], "aspects": [], "locks": [], "hierarchyPath": [ { "assetId": "{assetId}", "name": "{tenantName}" } ], "deleted": null, "_links": { "self": { "href": "{link}" }, "aspects": { "href": "{link}" }, "variables": { "href": "{link}" }, "location": { "href": "{link}" }, "parent": { "href": "{link}" } } } ``` The response shows that the `color` property for `Valve-001` is set to `gray`. ## Overwriting the Default Value of an Asset Change the `color` property from `gray` to `white` using the following request - replace `{id}` by the asset ID: ```http PATCH /assets/{id} ``` Provide the `{CurrentEtagValue}` for the `If-Match` parameter and the following JSON structure to define the asset: ```json { "variables": [ { "name": "color", "value": "white" } ] } ``` Verify the result in the response body: ```json { "assetId": "{assetId}", "tenantId": "{tenantId}", "name": "Valve-001", "etag": "{etagValue}", "externalId": "SN 123456-123-123456", "t2Tenant": null, "subTenant": null, "description": "Valve 001 installed somewhere", "timezone": "Europe/Berlin", "parentId": "{parentAssetId}", "typeId": "{tenantName}.Valve", "location": null, "fileAssignments": [], "variables": [ { "name": "color", "value": "white" } ], "aspects": [], "locks": [], "hierarchyPath": [ { "assetId": "{assetId}", "name": "{tenantName}" } ], "deleted": null, "_links": { "self": { "href": "{link}" }, "aspects": { "href": "{link}" }, "variables": { "href": "{link}" }, "location": { "href": "{link}" }, "parent": { "href": "{link}" } } } ``` ## Resetting the Default Value of an Asset When a default property is removed from the instance, it is automatically reset to the default value. Remove the asset's properties using the following request - replace `{id}` by the asset ID: ```http PATCH /asset/{id} ``` Provide the `{CurrentEtagValue}` for the `If-Match` parameter and the following JSON structure to define the asset: ```json { "variables": [ ] } ``` The return validates that the default value is restored in the asset instance: ```json { "assetId": "{assetId}", "tenantId": "{tenantId}", "name": "Valve-001", "etag": "{etagValue}", "externalId": "SN 123456-123-123456", "t2Tenant": null, "subTenant": null, "description": "Valve 001 installed somewhere", "timezone": "Europe/Berlin", "parentId": "{parentAssetId}", "typeId": "{tenantName}.Valve", "location": null, "fileAssignments": [], "variables": [ { "name": "color", "value": "gray" } ], "aspects": [], "locks": [], "hierarchyPath": [ { "assetId": "{assetId}", "name": "{tenantName}" } ], "deleted": null, "_links": { "self": { "href": "{link}" }, "aspects": { "href": "{link}" }, "variables": { "href": "{link}" }, "location": { "href": "{link}" }, "parent": { "href": "{link}" } } } ``` ## Changing the Default Value of Multiple Assets You can change the default value of already existing assets of the same asset type by updating the asset type. This automatically overwrites the default values of the derived assets. Use the following request - replace `{id}` by the `{tenantName}.Valve`, where `{tenantName}` is your tenant's name: ```http PATCH /assettypes/{id} ``` Provide the `{currentEtagValue}` for the `If-Match` parameter and the following JSON structure to define the asset type: ```json { "variables": [ { "name": "color", "dataType": "STRING", "searchable": true, "length": 10, "defaultValue": "black" } ] } ``` Note You need to add the complete `"variables"` instance for PATCH. Use the following request - replace `{assetID}` by your asset's ID: ```http GET /assets/{assetID} ``` In the response the color is changed to black: ```json { "assetId": "{assetId}", "tenantId": "{tenantId}", "name": "Valve-001", "etag": "{etagValue}", "externalId": "SN 123456-123-123456", "t2Tenant": null, "subTenant": null, "description": "Valve 001 installed somewhere", "timezone": "Europe/Berlin", "parentId": "{parentAssetId}", "typeId": "{tenantName}.Valve", "location": null, "fileAssignments": [], "variables": [ { "name": "color", "value": "black" } ], "aspects": [], "locks": [], "hierarchyPath": [ { "assetId": "{assetId}", "name": "{tenantName}" } ], "deleted": null, "_links": { "self": { "href": "{link}" }, "aspects": { "href": "{link}" }, "variables": { "href": "{link}" }, "location": { "href": "{link}" }, "parent": { "href": "{link}" } } } ``` ## Finding out the parentID of an Asset `parentId` is the ID of the asset one hierarchy level above the actual asset. In the example above the root asset was used as the parent. Inquire the root asset's ID using the following request: ```http GET /assets/root ``` If the parent is not the root asset then you can see the direct parent of an asset in the \`"hierarchyPath" list. It's always the last item in the list. This is also known as "breadcrumbs". When creating a new asset you need to add the parent asset's ID in the payload. If you do not remember it you can query it using the endpoint shown below and filter for any attribute you know. ```http GET /assets ``` # Asset Management Service – Modeling Assets This example illustrates how assets can be modeled and structured. In this example, a global wind power company, which has multiple wind farms around the globe, creates their first assets in Insights Hub. For more information on the endpoints used in the example, refer to the [API specification](api-assetmanagement-api.html). The company has one type of wind turbines, the `WindTurbine1000`, from which the following electrical and mechanical metrics are to be collected: - Power output in MW - Voltage - Rotations per minutes in RPM - Vibrations of rotors in Hz The company has wind farms installed in the North Sea and the Baltic Sea. For each wind farm, two assets shall be created. | | Wind Farm North Sea | Wind Farm Baltic Sea | | --------------- | ------------------- | -------------------- | | GPS coordinates | 54.008333, 6.598333 | 54.834, 14.068 | | Turbine #1 | N-001 | B-001 | | Turbine #2 | N-002 | B-002 | ## Prerequisites - You have access to a Insights Hub tenant - You have the `mdsp:core:assetmanagement.admin` role ## Preparing Templates Note {tenantName} used in below examples is the name of tenant, can be obtained using TenantManagements /tenantInfo API [TenantManagement](https://documentation.mindsphere.io/MindSphere/apis-xcelerator/core-tenantmanagement/api-tenantmanagement-api.html). ### Creating Aspect Types Two aspect types shall be created, one for electrical metrics and the other for mechanical metrics. This is done using the endpoint: ```http PUT /aspecttypes/{id} ``` For the aspect type for electrical metrics, the `{id}` is named `{tenantName}.ElectricalMetrics` and the body of the request is: ```json { "name": "ElectricalMetrics", "category": "dynamic", "scope": "private", "description": "Electrical metrics for wind turbines", "variables": [ { "name": "PWR_OUT", "dataType": "DOUBLE", "unit": "MW", "qualityCode": true, "searchable" : true }, { "name": "V_OUT", "dataType": "DOUBLE", "unit": "Volt", "qualityCode": true, "searchable" : true } ] } ``` The result can be verified by checking the response: ```json { "id": "{tenantName}.ElectricalMetrics", "tenantId": "{tenantId}", "name": "ElectricalMetrics", "category": "dynamic", "scope": "private", "description": "Electrical metrics for wind turbines", "variables": [ { "name": "PWR_OUT", "unit": "MW", "searchable": true, "qualityCode": true, "defaultValue": null, "dataType": "DOUBLE", "length": null }, { "name": "V_OUT", "unit": "Volt", "searchable": true, "qualityCode": true, "defaultValue": null, "dataType": "DOUBLE", "length": null } ], "etag": {etagvalue}, "_links": { "self": { "href": "{link}" } } } ``` For the aspect type for mechanical metrics, the `{id}` is named `{tenantName}.MechanicalMetrics` and the body of the request is: ```json { "name": "MechanicalMetrics", "category": "dynamic", "scope": "private", "description": "Mechanical metrics for wind turbines", "variables": [ { "name": "V", "dataType": "INT", "unit": "RPM", "qualityCode": true, "searchable" : true }, { "name": "VIB", "dataType": "INT", "unit": "Hz", "qualityCode": true, "searchable" : true } ] } ``` The created aspect type can be verified by checking the response: ```json { "id": "{tenantName}.MechanicalMetrics", "tenantId": "{tenantId}", "name": "MechanicalMetrics", "category": "dynamic", "scope": "private", "description": "Mechanical metrics for wind turbines", "variables": [ { "name": "V", "unit": "RPM", "searchable": true, "qualityCode": true, "defaultValue": null, "dataType": "INT", "length": null }, { "name": "VIB", "unit": "Hz", "searchable": true, "qualityCode": true, "defaultValue": null, "dataType": "INT", "length": null } ], "etag": {etagValue}, "_links": { "self": { "href": "{link}" } } } ``` ### Creating Asset Types After creating the required aspect types, the asset type for the `WindTurbine1000` can be created using the following endpoint: ```http PUT /assettypes/{id} ``` Here, the `{id}` is named `{tenantName}.WT1000` and the body of the request is: ```json { "name": "WindTurbine 1000", "description": "WindTurbine 1000 with best reliability.", "parentTypeId": "core.basicdevice", "instantiable": true, "scope": "private", "variables": [ { "name": "InstallationDate", "dataType": "TIMESTAMP", "searchable": true, "defaultValue": "1970-01-01T09:00:00+00:00" } ], "aspects": [ { "name": "ElectricalMetrics", "aspectTypeId": "{tenantName}.ElectricalMetrics" }, { "name": "MechanicalMetrics", "aspectTypeId": "{tenantName}.MechanicalMetrics" } ] } ``` The created asset type can be verified by checking the response: Response ```json { "parentTypeId": "core.basicdevice", "instantiable": true, "tenantId": "{tenantId}", "name": "WindTurbine 1000", "description": "WindTurbine 1000 with best reliability.", "scope": "private", "variables": [ { "name": "InstallationDate", "unit": null, "searchable": true, "defaultValue": "1970-01-01T09:00:00+00:00", "dataType": "TIMESTAMP", "length": null } ], "aspects": [ { "name": "ElectricalMetrics", "aspectType": { "id": "{tenantName}.ElectricalMetrics", "tenantId": "{tenantId}", "name": "ElectricalMetrics", "category": "dynamic", "scope": "private", "description": "Electrical metrics for wind turbines", "variables": [ { "name": "PWR_OUT", "unit": "MW", "searchable": true, "qualityCode": true, "defaultValue": null, "dataType": "DOUBLE", "length": null }, { "name": "V_OUT", "unit": "Volt", "searchable": true, "qualityCode": true, "defaultValue": null, "dataType": "DOUBLE", "length": null } ], "etag": {etagValue}, "_links": { "self": { "href": "{link}" } } } }, { "name": "MechanicalMetrics", "aspectType": { "id": "{tenantName}.MechanicalMetrics", "tenantId": "{tenantId}", "name": "MechanicalMetrics", "category": "dynamic", "scope": "private", "description": "Mechanical metrics for wind turbines", "variables": [ { "name": "V", "unit": "RPM", "searchable": true, "qualityCode": true, "defaultValue": null, "dataType": "INT", "length": null }, { "name": "VIB", "unit": "Hz", "searchable": true, "qualityCode": true, "defaultValue": null, "dataType": "INT", "length": null } ], "etag": {etagValue}, "_links": { "self": { "href": "{link}" } } } } ], "fileAssignments": [], "etag": {etagValue}, "_links": { "self": { "href": "{link}" }, "parent": { "href": "{link}" } }, "id": "{tenantName}.WT1000" } ``` ## Modeling Assets ### Creating Asset Sites For reducing complexity, an asset of type `basic.site` is created for each wind farm. This is done using the following endpoint: ```http POST /assets ``` For the wind farm in the North Sea, the body is: ```json { "name": "North Sea", "description": "Wind Farm North Sea", "location": { "longitude": 54.008333, "latitude": 6.598333 }, "typeId": "core.basicarea", "parentId": "{parentAssetId}", "timezone": "Europe/Berlin" } ``` The response of the call contains the newly created North Sea site asset so you can validate the correct creation: ```json { "assetId": "{assetId}", "tenantId": "{tenantId}", "name": "North Sea", "etag": {etagValue}, "externalId": null, "t2Tenant": null, "subTenant": null, "description": "Wind Farm North Sea", "timezone": "Europe/Berlin", "parentId": "{parentAssetId}", "typeId": "core.basicarea", "location": { "country": null, "region": null, "locality": null, "streetAddress": null, "postalCode": null, "longitude": 54.008333, "latitude": 6.598333 }, "fileAssignments": [], "variables": [], "aspects": [], "locks": [], "hierarchyPath": [ { "assetId": "{assetId}", "name": "{tenantName}" } ], "deleted": null, "_links": { "self": { "href": "{link}" }, "aspects": { "href": "{link}" }, "variables": { "href": "{link}" }, "location": { "href": "{link}" }, "parent": { "href": "{link}" } } } ``` The `{parentAssetId}` is the root asset. Its ID can be requested using the following endpoint: ```http GET /assets/root ``` For the wind farm in the Baltic Sea, the body is: ```json { "name": "Baltic Sea", "description": "Wind Farm Baltic Sea", "location": { "longitude": 54.834, "latitude": 14.068 }, "typeId": "core.basicarea", "parentId": "{parentAssetId}", "timezone": "Europe/Berlin" } ``` The response of the call contains the newly created Baltic Sea site asset so you can validate the correct creation: Response ```json { "assetId": "{assetId}", "tenantId": "{tenantId}", "name": "Baltic Sea", "etag": 0, "externalId": null, "t2Tenant": null, "subTenant": null, "description": "Wind Farm Baltic Sea", "timezone": "Europe/Berlin", "parentId": "{parentAssetId}", "typeId": "core.basicarea", "location": { "country": null, "region": null, "locality": null, "streetAddress": null, "postalCode": null, "longitude": 54.834, "latitude": 14.068 }, "fileAssignments": [], "variables": [], "aspects": [], "locks": [], "hierarchyPath": [ { "assetId": "{parentAssetId}", "name": "{tenantName}" } ], "deleted": null, "_links": { "self": { "href": "{link}" }, "aspects": { "href": "{link}" }, "variables": { "href": "{link}" }, "location": { "href": "{link}" }, "parent": { "href": "{link}" } } } ``` ### Creating Assets After the asset sites are set up, the actual assets are created using the following endpoint: ```http POST /assets/ ``` The body for the first asset for the wind farm of the North Sea is: ```json { "name": "N-001", "description": "Wind turbine 001 in North Sea", "variables": [ { "name": "InstallationDate", "value": "2018-08-28T09:00:00+00:00" } ], "typeId": "{tenantName}.WT1000", "parentId": "{AssetIdNorthSea}" } ``` The `{AssetIdNorthSea}` is the asset ID of the North Sea site asset. The response of the call contains the newly created asset for the wind farm in the North Sea, so you can validate the correct creation: Response ```json { "assetId": "{assetId}", "tenantId": "{tenantId}", "name": "N-001", "etag": {etagValue}, "externalId": null, "t2Tenant": null, "subTenant": null, "description": "Wind turbine 001 in North Sea", "timezone": "Europe/Berlin", "parentId": "{AssetIdNorthSea}", "typeId": "{tenantName}.WT1000", "location": { "country": null, "region": null, "locality": null, "streetAddress": null, "postalCode": null, "longitude": 54.008333, "latitude": 6.598333 }, "fileAssignments": [], "variables": [ { "name": "InstallationDate", "value": "2018-08-28T09:00:00+00:00" } ], "aspects": [], "locks": [], "hierarchyPath": [ { "assetId": "{rootAssetId}", "name": "{tenantName}" }, { "assetId": "{AssetIdNorthSea}", "name": "North Sea" } ], "deleted": null, "_links": { "self": { "href": "{link}" }, "aspects": { "href": "{link}" }, "variables": { "href": "{link}" }, "location": { "href": "{link}" }, "parent": { "href": "{link}" } } } ``` The response shows that value of the type variable `InstallationDate` of the asset is set to the value given in the request body. The default value defined by the asset type has been overwritten.\ The asset has inherited its location from the North Sea site asset.\ The asset has instantiated the aspects defined by its asset type `WK1000`, but the aspect list is empty. This is because all its aspects are dynamic and aspect lists are only populated on asset level, if the aspect is defined as static. The bodies for creating the other three assets are shown below: ```json { "name": "N-002", "description": "Wind turbine 002 in North Sea", "variables": [ { "name": "InstallationDate", "value": "2018-08-28T09:00:00+00:00" } ], "typeId": "{tenantName}.WT1000", "parentId": "{AssetIdNorthSea}" } ``` ```json { "name": "B-001", "description": "Wind turbine 001 in Baltic Sea", "variables": [ { "name": "InstallationDate", "value": "2018-08-28T09:00:00+00:00" } ], "typeId": "{tenantName}.WT1000", "parentId": "{AssetIdNorthSea}" } ``` ```json { "name": "B-002", "description": "Wind turbine 002 in Baltic Sea", "variables": [ { "name": "InstallationDate", "value": "2018-08-28T09:00:00+00:00" } ], "typeId": "{tenantName}.WT1000", "parentId": "{AssetIdNorthSea}" } ``` Attention The `parentId` for the wind turbines in the Baltic Sea is intentionally set wrong for this example. The next section is based on this. ### Moving assets In the code given above the wind turbines `B-001` and `B-002` are inherited from the North Sea asset site instead of the Baltic Sea asset site. This can be corrected by moving both assets using the move endpoint: ```http POST /assets/{id}/move ``` The request requires the following parameters: - `If-Match`: Has to be set as the `{etagValue}` of the asset. The `{etagValue}` is set to zero upon asset creation and incremented after each successful modification of the asset. - `id`: Has to be set as the `assetId` of the asset. It is listed in the response when an asset is created. - `moveParameters`: ```json { "newParentId": "{AssetIdBalticSea}" } ``` The `{AssetIdBalticSea}` is the asset ID of the Baltic Sea site asset. After performing the move operation, the location of `B-001` and `B-002` is correct. Hint If you forgot the individual asset IDs, you can get the list of all assets and their IDs by calling the following endpoint: ```http GET /assets ``` You can reduce the results using the [`filter` parameter](api-assetmanagement-references-filtering.html): ```json {"name": "Baltic Sea"} ``` # Asset Management Service – Update Asset Type This example illustrates how asset types variables can be updated or removed. In this example, a global wind power company which has multiple wind farms around the globe, creates their assets in Insights Hub. For more information on the endpoints used in the example, refer to the [API specification](api-assetmanagement-api.html). Note {tenantName} used in below examples is the name of tenant, can be obtained using TenantManagements /tenantInfo API [TenantManagement](https://documentation.mindsphere.io/MindSphere/apis-xcelerator/core-tenantmanagement/api-tenantmanagement-api.html). The company has one type of wind turbines, the `WindTurbine1000`, from which the electrical and mechanical metrics are collected. There are some static variables and aspects defined for the `WindTurbine1000`. Refer to the [Modeling Assets](api-assetmanagement-samples-modelingassets.html). After creating the required aspect types, the asset type for the `WindTurbine1000` can be created using the following endpoint: ```http PUT /assettypes/{id} ``` Here, the `{id}` is named `{tenantName}.WindTurbine1000` and the body of the request is: Request ```json { "name": "WindTurbine1000", "description": "WindTurbine 1000 with best reliability.", "parentTypeId": "core.basicdevice", "instantiable": true, "scope": "private", "variables": [ { "name": "Installation", "dataType": "TIMESTAMP", "searchable": true, "defaultValue": "1970-01-01T09:00:00+00:00" }, { "name": "Manufacturer", "dataType": "STRING", "searchable": true, "length": 8, "defaultValue": "CompanyA" }, { "name": "Type", "dataType": "STRING", "searchable": true, "length": 12, "defaultValue": "ScrewTurbine" }, { "name": "Weight", "dataType": "INT", "searchable": true, "defaultValue": 20 } ], "aspects": [ { "name": "ElectricalMetrics", "aspectTypeId": "{{tenantName}}.ElectricalMetrics" }, { "name": "MechanicalMetrics", "aspectTypeId": "{{tenantName}}.MechanicalMetrics" } ] } ``` The created asset type can be verified by checking the response: Response ```json { "parentTypeId": "core.basicdevice", "instantiable": true, "tenantId": "{tenantId}", "name": "WindTurbine1000", "description": "WindTurbine 1000 with best reliability.", "scope": "private", "variables": [ { "name": "Installation", "unit": null, "searchable": true, "dataType": "TIMESTAMP", "length": null, "defaultValue": "1970-01-01T09:00:00+00:00" }, { "name": "Manufacturer", "unit": null, "searchable": true, "dataType": "STRING", "length": 8, "defaultValue": "CompanyA" }, { "name": "Type", "unit": null, "searchable": true, "dataType": "STRING", "length": 12, "defaultValue": "ScrewTurbine" }, { "name": "Weight", "unit": null, "searchable": true, "dataType": "INT", "length": null, "defaultValue": "20" } ], "aspects": [ { "name": "ElectricalMetrics", "aspectType": { "id": "{tenantName}.ElectricalMetrics", "tenantId": "{tenantId}", "name": "ElectricalMetrics", "category": "dynamic", "scope": "private", "description": "Electrical metrics for wind turbines", "variables": [ { "name": "PWR_OUT", "unit": "MW", "searchable": true, "qualityCode": true, "defaultValue": null, "dataType": "DOUBLE", "length": null }, { "name": "V_OUT", "unit": "Volt", "searchable": true, "qualityCode": true, "defaultValue": null, "dataType": "DOUBLE", "length": null } ], "etag": 0, "sharing": { "modes": [] }, "_links": { "self": { "href": "https://gateway.eu1.mindsphere.io/api/assetmanagement/v3/aspecttypes/{tenantName}.ElectricalMetrics" } } } }, { "name": "MechanicalMetrics", "aspectType": { "id": "{tenantName}.MechanicalMetrics", "tenantId": "{tenantId}", "name": "MechanicalMetrics", "category": "dynamic", "scope": "private", "description": "Mechanical metrics for wind turbines", "variables": [ { "name": "V", "unit": "RPM", "searchable": true, "qualityCode": true, "defaultValue": null, "dataType": "INT", "length": null }, { "name": "VIB", "unit": "Hz", "searchable": true, "qualityCode": true, "defaultValue": null, "dataType": "INT", "length": null } ], "etag": 0, "sharing": { "modes": [] }, "_links": { "self": { "href": "https://gateway.eu1.mindsphere.io/api/assetmanagement/v3/aspecttypes/{tenantName}.MechanicalMetrics" } } } } ], "fileAssignments": [], "sharing": { "modes": [] }, "etag": 0, "_links": { "self": { "href": "https://gateway.eu1.mindsphere.io/api/assetmanagement/v3/assettypes/{tenantName}.WindTurbine1000" }, "parent": { "href": "https://gateway.eu1.mindsphere.io/api/assetmanagement/v3/assettypes/core.basicdevice" } }, "id": "{tenantName}.WindTurbine1000" } ``` ## Deleting Variables The asset type `{tenantName}.WindTurbine1000` has four static variables, the variable `Weight` needs to be removed from the asset type. It can be removed by using the Asset Type update API as below: ```http PUT /assettypes/{id} ``` Here, the `{id}` is named `{tenantName}.WindTurbine1000` and the body of the request is as below with the removed variable `Weight`, the `If-Match` header needs to have the value of `etag` from the Asset Type response. Request ```json { "name": "WindTurbine1000", "description": "WindTurbine 1000 with best reliability.", "parentTypeId": "core.basicdevice", "instantiable": true, "scope": "private", "variables": [ { "name": "Installation", "dataType": "TIMESTAMP", "searchable": true, "defaultValue": "1970-01-01T09:00:00+00:00" }, { "name": "Manufacturer", "dataType": "STRING", "searchable": true, "length": 8, "defaultValue": "CompanyA" }, { "name": "Type", "dataType": "STRING", "searchable": true, "length": 12, "defaultValue": "ScrewTurbine" } ], "aspects": [ { "name": "ElectricalMetrics", "aspectTypeId": "{{tenantName}}.ElectricalMetrics" }, { "name": "MechanicalMetrics", "aspectTypeId": "{{tenantName}}.MechanicalMetrics" } ] } ``` The updated asset type can be verified by checking the response, the variable is removed from the response and also from its Asset Instances: Response ```json { "parentTypeId": "core.basicdevice", "instantiable": true, "tenantId": "{tenantId}", "name": "WindTurbine1000", "description": "WindTurbine 1000 with best reliability.", "scope": "private", "variables": [ { "name": "Installation", "unit": null, "searchable": true, "dataType": "TIMESTAMP", "length": null, "defaultValue": "1970-01-01T09:00:00+00:00" }, { "name": "Manufacturer", "unit": null, "searchable": true, "dataType": "STRING", "length": 8, "defaultValue": "CompanyA" }, { "name": "Type", "unit": null, "searchable": true, "dataType": "STRING", "length": 12, "defaultValue": "ScrewTurbine" } ], "aspects": [ { "name": "ElectricalMetrics", "aspectType": { "id": "{tenantName}.ElectricalMetrics", "tenantId": "{tenantId}", "name": "ElectricalMetrics", "category": "dynamic", "scope": "private", "description": "Electrical metrics for wind turbines", "variables": [ { "name": "PWR_OUT", "unit": "MW", "searchable": true, "qualityCode": true, "defaultValue": null, "dataType": "DOUBLE", "length": null }, { "name": "V_OUT", "unit": "Volt", "searchable": true, "qualityCode": true, "defaultValue": null, "dataType": "DOUBLE", "length": null } ], "etag": 0, "sharing": { "modes": [] }, "_links": { "self": { "href": "https://gateway.eu1.mindsphere.io/api/assetmanagement/v3/aspecttypes/{tenantName}.ElectricalMetrics" } } } }, { "name": "MechanicalMetrics", "aspectType": { "id": "{tenantName}.MechanicalMetrics", "tenantId": "{tenantId}", "name": "MechanicalMetrics", "category": "dynamic", "scope": "private", "description": "Mechanical metrics for wind turbines", "variables": [ { "name": "V", "unit": "RPM", "searchable": true, "qualityCode": true, "defaultValue": null, "dataType": "INT", "length": null }, { "name": "VIB", "unit": "Hz", "searchable": true, "qualityCode": true, "defaultValue": null, "dataType": "INT", "length": null } ], "etag": 0, "sharing": { "modes": [] }, "_links": { "self": { "href": "https://gateway.eu1.mindsphere.io/api/assetmanagement/v3/aspecttypes/{tenantName}.MechanicalMetrics" } } } } ], "fileAssignments": [], "sharing": { "modes": [] }, "etag": 1, "_links": { "self": { "href": "https://gateway.eu1.mindsphere.io/api/assetmanagement/v3/assettypes/{tenantName}.WindTurbine1000" }, "parent": { "href": "https://gateway.eu1.mindsphere.io/api/assetmanagement/v3/assettypes/core.basicdevice" } }, "id": "{tenantName}.WindTurbine1000" } ``` ## Updating Variables The asset type `{tenantName}.WindTurbine1000` has three static variables defined. Some updates are required on the variables as below: - The variable `Manufacturer` needs to be renamed to `Vendor`, the `defaultValue` needs to be changed to `VendorZ` and its length needs to be increased to `20`. - The variable `Type` needs to be renamed to `Volume` and unit needs to be changed to `cc`. It can be done by using below asset type variables update API: ```http PATCH /assettypes/{id}/variables ``` Here, the `{id}` is `{tenantName}.WindTurbine1000` and the body of the request is a Variable Dictionary Map as below. The `If-Match` header in the request needs to have the value of `etag` of asset type. The Content-Type of the request has to be `application/merge-patch+json`. Only the variables and properties present in the payload will be updated and the other variables and its properties will not be updated. In case of validation failure, the request is failed without any updates. ```json { "Manufacturer": { "name": "Vendor", "defaultValue": "VendorZ", "length": 20 }, "Type": { "name": "Volume", "unit": "cc" } } ``` Successful Response: ```json 204 No Content ``` Response Header: ```json ETag : 1 ``` This is the etag of asset type. It can be used to further update asset type and its variables again. The updated asset type can be re-verified by checking the response asset type. ```http GET /assettypes/{id} ``` Here, the `{id}` is named `{tenantName}.WindTurbine1000`. Response ```json { "parentTypeId": "core.basicdevice", "instantiable": true, "tenantId": "{tenantId}", "name": "WindTurbine1000", "description": "WindTurbine 1000 with best reliability.", "scope": "private", "variables": [ { "name": "Installation", "unit": null, "searchable": true, "dataType": "TIMESTAMP", "length": null, "defaultValue": "1970-01-01T09:00:00+00:00" }, { "name": "Vendor", "unit": null, "searchable": true, "dataType": "STRING", "length": 20, "defaultValue": "VendorZ" }, { "name": "Volume", "unit": "cc", "searchable": true, "dataType": "STRING", "length": 12, "defaultValue": "ScrewTurbine" } ], "aspects": [ { "name": "ElectricalMetrics", "aspectType": { "id": "{tenantName}.ElectricalMetrics", "tenantId": "{tenantId}", "name": "ElectricalMetrics", "category": "dynamic", "scope": "private", "description": "Electrical metrics for wind turbines", "variables": [ { "name": "PWR_OUT", "unit": "MW", "searchable": true, "qualityCode": true, "defaultValue": null, "dataType": "DOUBLE", "length": null }, { "name": "V_OUT", "unit": "Volt", "searchable": true, "qualityCode": true, "defaultValue": null, "dataType": "DOUBLE", "length": null } ], "etag": 0, "sharing": { "modes": [] }, "_links": { "self": { "href": "https://gateway.eu1.mindsphere.io/api/assetmanagement/v3/aspecttypes/{tenantName}.ElectricalMetrics" } } } }, { "name": "MechanicalMetrics", "aspectType": { "id": "{tenantName}.MechanicalMetrics", "tenantId": "{tenantId}", "name": "MechanicalMetrics", "category": "dynamic", "scope": "private", "description": "Mechanical metrics for wind turbines", "variables": [ { "name": "V", "unit": "RPM", "searchable": true, "qualityCode": true, "defaultValue": null, "dataType": "INT", "length": null }, { "name": "VIB", "unit": "Hz", "searchable": true, "qualityCode": true, "defaultValue": null, "dataType": "INT", "length": null } ], "etag": 0, "sharing": { "modes": [] }, "_links": { "self": { "href": "https://gateway.eu1.mindsphere.io/api/assetmanagement/v3/aspecttypes/{tenantName}.MechanicalMetrics" } } } } ], "fileAssignments": [], "sharing": { "modes": [] }, "etag": 2, "_links": { "self": { "href": "https://gateway.eu1.mindsphere.io/api/assetmanagement/v3/assettypes/{tenantName}.WindTurbine1000" }, "parent": { "href": "https://gateway.eu1.mindsphere.io/api/assetmanagement/v3/assettypes/core.basicdevice" } }, "id": "{tenantName}.WindTurbine1000" } ``` The above example illustrates that multiple variables can be updated using update variables API. Interchanging of variable names are not allowed in single request. This means that the name of existing variable should not be used to rename the another variable in the same request. In this scenario, below request payload will result in a failure: ```json { "Manufacturer": { "name": "Type" }, "Type": { "name": "Volume" } } ``` # Notification for Asset Model Changes ## Overview With subscription-based notification, the 3rd party applications can now integrate subscriptions to topics related to Asset Model. Such applications would be then able to receive notifications when there's a change in Asset Model. Applications can respond with desired actions upon receiving a notification such as using `GET aspect by ID API`, `GET asset type by ID API` or `GET asset by ID API` to fetch more details about the respective asset model object, if required. This feature enables developers to build optimized interactions in the applications and make them performant. Moreover, this improves the user experience for the developers. This mechanism enables customers when to poll the Asset Model data. The notification will be received on the following changes to Asset Model objects: ### Aspect Type - Create Aspect Type - Update Aspect Type - Change description - Add variable on aspect type - Remove variable from aspect type - Rename variable on aspect type - Change unit on aspect variable - Change default value on static aspect variable - Delete Aspect Type ### Asset Type - Create Asset Type - Update Asset Type - Change description, image on Asset Type - Rename Asset Type - Add Asset Type direct variable - Rename Asset Type direct variable - Change unit on Asset Type direct variable - Change default value on Asset Type direct variable - Change length on Asset Type direct variable of type STRING - Delete Asset Type direct Variable - Create Aspect on Asset Type - Remove Aspect from Asset Type - Rename Aspect on Asset Type - Delete Asset Type ### Asset - Create Asset - Update Asset - Change asset name, description, and location - Change or override the default value of the direct asset type variable - Change or override the default value of the static aspect variable - Delete Asset - Move Asset To create and publish App for consuming Asset Model notifications, refer to [How to Subscribe to notifications for Asset Model change](https://documentation.mindsphere.io/MindSphere/howto/howto-receive-notifications.html) ## Limitation - The underlying messaging service may bring nomenclature/version changes in the near term; which may need changes and adaption in the customer application. ## References - [Message Broker API](../core-messagebroker/api-messagebroker-overview.html) - Provisioning API ## FAQs 1. What is the notification level in the Asset Model? - The notification will be available at the tenant level. 1. What is the maximum number of asset model resource information per notification? - The notification will be real-time on an Asset Model change and will be a single message per action. 1. Can customers get delayed notification? - Yes, in case any Insights Hub component is non-operational. # Asset Notifications ## Notification for aspect type create or update or delete - When aspect types are created or updated or deleted in system, user will receive notification. - The notification message will contain information like id of the aspect type, resource, action, tenant and timestamp. The tenant and topicId will be provided as part of header for the message. Sample message notification when aspect types are created or updated or deleted: ```json { "header": { "topic": "mdsp.core.assetmanagement.v1.pubsub.aspecttype.ext.notification", "tenantContext": [ "siemens" ] }, "aspectType": { "id": "siemens.aspectType", "resource": "ASPECT_TYPE", "action": "CREATE", "tenant": "siemens", "timestamp": "2022-02-08T06:24:35.629Z" } } ``` ## Notification for asset type create or update or delete - When asset types are created or updated or deleted in system, user will receive notification. - The notification message will contain information like id of the asset type, resource, action, tenant and timestamp. The tenant and topicId will be provided as part of header for the message. Sample message notification when asset types are created or updated or deleted: ```json { "header": { "topic": "mdsp.core.assetmanagement.v1.pubsub.assettype.ext.notification", "tenantContext": [ "siemens" ] }, "assetType": { "id": "siemens.assetType", "resource": "ASSET_TYPE", "action": "UPDATE", "tenant": "siemens", "timestamp": "2022-02-08T06:24:35.629Z" } } ``` ## Notification for asset create or update or delete - When assets are created or updated or deleted in system, user will receive notification. - The notification message will contain information like id of the asset, resource, action, tenant and timestamp. The tenant and topicId will be provided as part of header for the message. Sample message notification when assets are created or updated or deleted: ```json { "header": { "topic": "mdsp.core.assetmanagement.v1.pubsub.asset.ext.notification", "tenantContext": [ "siemens" ] }, "asset": { "id": "4ed7e69d1f6747dcb96dafec7a844488", "resource": "ASSET", "action": "DELETE", "tenant": "siemens", "timestamp": "2022-02-08T06:24:35.629Z" } } ``` # Case Management Service – API Overview ## Region Europe 1 [Download OpenAPI Specification](/insights-hub/api_specs/cases-v3-5-0-eu1-xcelerator.yaml) # Case Management Service – Filtering GET All cases endpoints with the parameter `filter` supports querying for *filterable* properties. When reading the cases, all cases are filtered for the main tenant. In addition, you can provide one or more field names in the filter query (all other field names will be ignored): - `handle` - `dueDate` - `notifyAssignee` - `title` - `type` - `status` - `assignedTo` - `description` - `source` - `priority` - `createdBy` - `createdDate` - `modifiedBy` - `modifiedDate` Wildcards are not supported. Timestamps must be in the following example format: `2023-01-01T00:00:00.00Z` When a filter has no matches, the response will be empty. ## Filter functions ### Function `eq` (equals) ```json { "status": "OPEN" } ``` or ```json { "status": { "eq": "OPEN" } } ``` ### Function `in` The function `in` is similar to `equals`, but instead matching to one value, it checks an array of values, and matches for any of them. ```json { "status": { "in": ["OPEN", "DONE"] } } ``` ### Function `notIn` The function `notIn` is reverse to `in`. It checks an array of values and matches for none of them. ```json { "status": { "notIn": ["OPEN", "DONE"] } } ``` ### Function `startsWith` ```json { "title": { "startsWith": "Rule" } } ``` ### Function `endsWith` ```json { "createdBy": { "endsWith": "@test.com" } } ``` ### Function `like` The function returns cases whose title includes `Rule` either as prefix or suffix. `%25` is encoded form of `%` ```json { "title": { "like": "%25Rule%25" } } ``` ### Function `notLike` The function `notLike` is reverse to `like`, returns the cases whose title includes `Rule` neither as prefix nor suffix. `%25` is encoded form of `%` ```json { "title": { "notLike": "%25Rule%25" } } ``` ### Function `gt` The function `gt` operates as *greater than the input value* as matching criteria to filter cases. ```json { "createdDate": { "gt": "2023-11-01T00:00:00.00Z" } } ``` ### Function `gte` The function `gte` operates as *greater than and equals to the input value* as matching criteria to filter cases. ```json { "createdDate": { "gte": "2023-11-01T00:00:00.00Z" } } ``` ### Function `lt` The function `lt` operates as *less than the input value* as matching criteria to filter cases. ```json { "createdDate": { "lt": "2023-11-01T00:00:00.00Z" } } ``` ### Function `lte` The function `lte` operates as *less than and equals to the input value* as matching criteria to filter cases. ```json { "createdDate": { "lte": "2023-11-01T00:00:00.00Z" } } ``` ### Function `between` The function `between` operates as *within the range of input values* as matching criteria to filter cases. ```json { "createdDate": { "between": ["2023-11-01T00:00:00.000Z", "2023-11-12T23:59:59.999Z"] } } ``` ### Function `notBetween` The function `notBetween` operates as *outside of the range of input values* as matching criteria to filter cases. ```json { "createdDate": { "notBetween": ["2023-11-01T00:00:00.000Z", "2023-11-12T23:59:59.999Z"] } } ``` ## Filter operations ### Operation `and` ```json { "status": "OPEN", "createdBy": {"endsWith": "@test.com"} } ``` or ```json { "status": { "in": ["OPEN","DONE","CANCELLED","ARCHIVED"] }, "createdBy": {"endsWith": "@test.com"} } ``` ## Nested fields ### Field `associations` You can filter cases for any associated input asset or event ID(s). E.g., all the cases that includes asset of ID `cb72dfd7400e4fc6a275f22e6751cce6` of type `ASSET` will have below request filter parameter json. ```json { "associations": [ { "type": "ASSET", "id": "cb72dfd7400e4fc6a275f22e6751cce61" } ] } ``` E.g., all the cases that includes multiple events of type `EVENT` will have below request filter parameter json. ```json { "associations": [ { "type": "EVENT", "id": "c72d22e6751cce6" }, { "type": "EVENT", "id": "b14d22e6751cce6" } ] } ``` Note Filtering cases based on nested fields `associations` matches for any asset/event type in the cases list for that specific tenant, the field `id` additionally matches with the given asset/event type. ### Field `externalSystems` You can filter cases for any associated input external systems name(s) based on multiple parameters. E.g., all the cases that includes type as `source` and name as `Senseye` and type as will have below request filter parameter json. ```json { "externalSystems": [ { "type": "source", "name": "Senseye" } ] } ``` # Case Management Service ## Idea “Cases” provide a basic digital workflow for work requests (maintenance, repair, inspection and incident handling) that are essential to monitor asset health and to detect technical issues before they lead to asset failure and downtime. You can create, track and update cases using an API. ## Access For accessing this service, you need to have the respective roles listed in [Case Management roles and scopes](https://documentation.mindsphere.io/MindSphere/concepts/concept-roles-scopes.html). ## Basics Case provides a ticketing way to plan and assign audits or maintenance of assets. Detailed case helps the assignee to prioritize and execute the task. Timely updated case provides the status overview of all cases. - **Title\***: A well-defined title should summarize the task or expectation. - **Description**: A more detailed description about the case should be added. - **Assigned to**: An assignment defines the person to work on the task. The assignee gets an email notification with the case link, if 'Notify Assignee' flag is provided as 'true'. - **Due Date\***: This date sets the expectation about getting the task completed. - **Status**: Case can be updated to have different statuses to reflect the progress. The following status are available: - OPEN: Default status of case at the time of creation. - INPROGRESS: A case is expected to be in this status as the work starts. - ONHOLD: A case can be marked as on hold using this status. - OVERDUE: System puts the case in overdue if it is not closed by the defined due date. - DONE: A case can be moved to this status as the work gets done. - CANCELLED: A case can be marked as cancelled using this status. - ARCHIVED: A case can be archived using this status. - **Priority**: The following priority options are available to prioritize the work items: - LOW - MEDIUM - HIGH - EMERGENCY - **Type**: Type of a case defines the reasoning behind the initiation of that case - PLANNED: This defines that case is a planned activity or scheduled activity to be done. - INCIDENT: This defines that case is an outcome of an incident happened and it is not a planned work. - ANNOTATION: This defines that case is an annotation activity to be performed. - **Association**: Cases can be associated with different entity types. Supported entities are ASSET or EVENT. e.g. ASSET. - **Attachment**: Attachments can be any files supporting the work to be done or the details captured while working. ## Features - Intuitive overview enables easy monitoring and prioritization of cases. - Due dates and assignments enable convenient tracking and planning. - Details can be easily added using attachments, associations and external systems. - By-default, GET All cases service will return 1000 cases per page, when page and size request parameters are not given in request as parameters by client. ## Limitations - A tenant can create maximum 676000 cases. You may delete older/unused/Done cases to maintain the total count within this limit. - A case can have a maximum of 10 attachments. You may delete attachments to attach new ones if total has reached this limit. - Currently, a case can be associated only with two entities of type ASSET or EVENT. A case can have maximum of 11 associations i.e. maximum 10 events and 1 asset can be linked with single case. - A case can have a maximum of 10 external systems. You may update external systems to attach new ones if total has reached this limit. ## Example Scenario Cases could be created for the following reasons: - Follow-up cases - to schedule a repair (at a later date) found during ongoing activity - Cases for incidents or reported problems (non-routine cases) - Planned work during scheduled plant shutdown - Equipment / Process modification projects - Cases for compliance of regulations and audits - Outcome of these work requests carry insights that could optimize operations, extend asset life, improve worker safety. # Case Management Service – Pagination When working with a tenant, it typically collects more and more cases. When exploring these using the respective `GET` /cases endpoint, the responses would contain object lists with several hundreds of entries. In order to break the results down into processable chunks, the Case Management Service provides a pagination functionality. This functionality divides the results into pages, which only hold a defined number of results. The following endpoints support the pagination functionality: ```http GET /cases ``` ## Pagination Parameters | Parameter | Description | Default Value | | --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------- | | `size` | Defines how many objects are returned per page. (optional) | 15 | | `page` | Each page only holds as many objects as defined by the `size` parameter and by default, only the first page is returned. Use this parameter to change the page number to be returned. (optional) | 0 | | `sort` | Defines based on which object attributes the results are sorted. (optional) | createdDate in descending order | As the table indicates, the pagination parameters are optional. By default, the endpoints return the first 15 elements in createdDate descending order. The page field of the response lists the value of the `size` parameter, `totalElements` as the total number of elements, `totalPages` as the total number of pages, and `number` as the page number of the returned page, for example: ```javascript "page": { "size": 15, "totalElements": 365, "totalPages": 25, "number": 0 } ``` ### Size The number of objects returned per page is set using the `size` parameter. Its default value is 15. Note that changing the `size` value affects the total number of pages, as the total number of objects is divided into bigger or smaller chunks. Maximum page size of 200 is allowed for case management GET APIs: GET /cases. ### Page By default, the first page of a result is returned. Its page number is 0, as is the default value of the `page` parameter. If a result consists of multiple pages, select the page to be returned using this parameter. ### Sorting Using the `sort` parameter, you can modify both the sorting attributes and the order of the sorting. The input for this parameter is case-sensitive and accepts the name of the object attribute to base the sorting on. All object attributes specified as filterable are allowed as input.\ For sorting in descending order, append the name of object attribute with `:desc` and append `:asc` for ascending order. This requires that the object attribute is set explicitly, even if you want to use the default value. Multiple sorting attributes with order can be given in request `sort` parameter. # Case Management Service – Samples ## Create Case The case is created using the following request: ```http POST /cases ``` The following JSON structure is given to define the case: ```json { "notifyAssignee": false, "dueDate": "2023-11-25", "title": "Floor Machine Maintenance", "priority": "LOW", "status": "OPEN", "type": "PLANNED", "description": "Dismantle and clean parts" } ``` ```json { "message": "Case created with case-handle AA-002", "handle": "AA-002" } ``` In the response, `handle` is a unique identifier for a given tenant and case which will be used for further operations such update/delete. ## Assignment and Notification While creating case, it can be assigned to an email address and optionally notify the assignee with email. ```http POST /cases ``` In the following request, the field descriptions are as below: - `assignedTo` is used to assign case to a user. The value of this field is the email address of the user. - `notifyAssignee`, if this field is set to true, then email notification is sent to the assigned email address. The Default value for this field is false. So, if case is assigned to a user, by default notification is not sent. ```json { "notifyAssignee": true, "assignedTo": "test@siemens.com", "dueDate": "2022-11-25", "title": "Floor Machine Maintenance", "priority": "LOW", "status": "OPEN", "type": "PLANNED", "description": "Dismantle and clean parts" } ``` ## Associations Associations are used to link entities to cases. Here, case service allows linking assets using assetId. In the following request, the field descriptions are as below: - `id` field which is id of entity to be associated. - `type` field is the type of entity to be associated. - Currently, only `ASSET` or `EVENT` type is supported, so the id is asset or event id. - Maximum 1 asset and 10 events can be associated with a case. ### Association for existing case To add associations to an existing case, use the following API with `handle`. ```http PATCH /cases/AA-000/associations ``` ```json { "associations": [ { "id": "cb72dfd7400e4fc6a275f22e6751cce6", "type": "ASSET" } ] } ``` Here id `cb72dfd7400e4fc6a275f22e6751cce6`is the asset to be linked to case. ### Association for new case To define association while creation of cases, add associations field to case request. Use the same API as create a case ```http POST /cases ``` In the following request, notice the association field ```json { "notifyAssignee": false, "dueDate": "2022-11-25", "title": "Floor Machine Maintenance", "priority": "LOW", "status": "OPEN", "type": "PLANNED", "description": "Dismantle and clean parts", "associations": [ { "id": "cb72dfd7400e4fc6a275f22e6751cce6", "type": "ASSET" }, { "id": "ad0e4fc6a275f22e6751", "type": "EVENT" } ] } ``` Here, id `cb72dfd7400e4fc6a275f22e6751cce6` is the asset and `ad0e4fc6a275f22e6751` is the event to be linked to case. ## Attachments Attachments field can be used to attach files to cases. Files stored in IoT File storage can be attached to a case. Let's assume the file we want to attach to case exists at `/api/iotfile/v3/files/cb72dfd7400e4fc6a275f22e6751cce6/AA-000/file.pdf`. In request below, observe the following fields according example above - `name` is the name of file, `file.pdf`. - `assetId` is the asset id where file is uploaded, `cb72dfd7400e4fc6a275f22e6751cce6`. - `path` is the relative path to the file from asset, `AA-000`. Refer [IoT File Service](../iot-iotfile/api-iotfile-overview.html) for more information about uploading files. ### Attachments for existing case To attach the above file on case `AA-000`, use the following API: ```http PATCH /cases/AA-000/attachments ``` ```json { "attachments": [ { "name": "file.pdf", "assetId": "cb72dfd7400e4fc6a275f22e6751cce6", "path": "AA-000" } ] } ``` ### Attachment for new case To define attachment while creation of cases, add attachments field to case request. Use same API as create case. ```http POST /cases ``` In the following request, notice the `attachments` field ```json { "notifyAssignee": false, "dueDate": "2022-11-25", "title": "Floor Machine Maintenance", "priority": "LOW", "status": "OPEN", "type": "PLANNED", "description": "Dismantle and clean parts", "attachments": [ { "name": "file.pdf", "assetId": "cb72dfd7400e4fc6a275f22e6751cce6", "path": "AA-000" } ] } ``` ### Related Links - [IoT File Service](../iot-iotfile/api-iotfile-overview.html) ## Get Aggregate Summary This API is useful for getting overview of cases in a tenant. ```http GET /cases/aggregate ``` The following is the sample response from above API: ```json { "statusInfo": { "OPEN": { "count": 1, "fieldName": "OPEN" }, "INPROGRESS": { "count": 0, "fieldName": "INPROGRESS" }, "ONHOLD": { "count": 0, "fieldName": "ONHOLD" }, "DONE": { "count": 0, "fieldName": "DONE" }, "OVERDUE": { "count": 1, "fieldName": "OVERDUE" }, "CANCELLED": { "count": 0, "fieldName": "CANCELLED" }, "ARCHIVED": { "count": 0, "fieldName": "ARCHIVED" } }, "priorityInfo": { "EMERGENCY": { "count": 0, "fieldName": "EMERGENCY" }, "MEDIUM": { "count": 0, "fieldName": "MEDIUM" }, "HIGH": { "count": 0, "fieldName": "HIGH" }, "LOW": { "count": 1, "fieldName": "LOW" } }, "severityByStatus": { "overview": { "OPEN": { "EMERGENCY": { "count": 0, "fieldName": "EMERGENCY" }, "MEDIUM": { "count": 0, "fieldName": "MEDIUM" }, "HIGH": { "count": 0, "fieldName": "HIGH" }, "LOW": { "count": 1, "fieldName": "LOW" } }, "INPROGRESS": { "EMERGENCY": { "count": 0, "fieldName": "EMERGENCY" }, "MEDIUM": { "count": 0, "fieldName": "MEDIUM" }, "HIGH": { "count": 0, "fieldName": "HIGH" }, "LOW": { "count": 0, "fieldName": "LOW" } }, "ONHOLD": { "EMERGENCY": { "count": 0, "fieldName": "EMERGENCY" }, "MEDIUM": { "count": 0, "fieldName": "MEDIUM" }, "HIGH": { "count": 0, "fieldName": "HIGH" }, "LOW": { "count": 0, "fieldName": "LOW" } }, "DONE": { "EMERGENCY": { "count": 0, "fieldName": "EMERGENCY" }, "MEDIUM": { "count": 0, "fieldName": "MEDIUM" }, "HIGH": { "count": 0, "fieldName": "HIGH" }, "LOW": { "count": 0, "fieldName": "LOW" } }, "OVERDUE": { "EMERGENCY": { "count": 0, "fieldName": "EMERGENCY" }, "MEDIUM": { "count": 0, "fieldName": "MEDIUM" }, "HIGH": { "count": 0, "fieldName": "HIGH" }, "LOW": { "count": 1, "fieldName": "LOW" } }, "CANCELLED": { "EMERGENCY": { "count": 0, "fieldName": "EMERGENCY" }, "MEDIUM": { "count": 0, "fieldName": "MEDIUM" }, "HIGH": { "count": 0, "fieldName": "HIGH" }, "LOW": { "count": 0, "fieldName": "LOW" } }, , "ARCHIVED": { "EMERGENCY": { "count": 0, "fieldName": "EMERGENCY" }, "MEDIUM": { "count": 0, "fieldName": "MEDIUM" }, "HIGH": { "count": 0, "fieldName": "HIGH" }, "LOW": { "count": 0, "fieldName": "LOW" } }, } }, "totalCases": 1 } ``` The `statusInfo` field retrieves the count of cases for a tenant by status. The `priorityInfo` field retrieves the count of cases for a tenant by priority. The `severityByStatus` field retrieves the count of cases for a tenant by status and then further segregated by severity. In the above example, there is only one case with LOW priority and OPEN status, so we can verify aggregate status. # Event Management - API Overview ## Region Europe 1 [Download OpenAPI Specification](/insights-hub/api_specs/eventmanagement-v3-17-0-xcelerator.yaml) # EventManagement - Usage quota and limits EventManagement APIs are restricted with following limits. | **API Endpoint** | **XS** | **S** | **M** | **L** | **XL** | | -------------------------------------------------- | ------ | ----- | ----- | ----- | ------ | | core_eventmanagement_v3_events_post_1m | 3000 | 3000 | 3000 | 3000 | 3000 | | core_eventmanagement_v3_events_post_1h | 20000 | 20000 | 20000 | 20000 | 20000 | | core_eventmanagement_v3_events_get_10s | 100 | 100 | 100 | 100 | 100 | | core_eventmanagement_v3_events_get_1m | 500 | 500 | 500 | 500 | 500 | | core_eventmanagement_v3_events_get_1h | 10000 | 10000 | 10000 | 10000 | 10000 | | core_eventmanagement_v3_events-~\_get_1m | 500 | 500 | 500 | 500 | 500 | | core_eventmanagement_v3_events-~\_get_1h | 6000 | 6000 | 6000 | 6000 | 6000 | | core_eventmanagement_v3_events-~\_put_1m | 500 | 500 | 500 | 500 | 500 | | core_eventmanagement_v3_events-~\_put_1h | 6000 | 6000 | 6000 | 6000 | 6000 | | core_eventmanagement_v3_eventTypes_post_1m | 200 | 200 | 200 | 200 | 200 | | core_eventmanagement_v3_eventTypes_post_1h | 1000 | 1000 | 1000 | 1000 | 1000 | | core_eventmanagement_v3_eventTypes_get_1m | 500 | 500 | 500 | 500 | 500 | | core_eventmanagement_v3_eventTypes_get_1h | 6000 | 6000 | 6000 | 6000 | 6000 | | core_eventmanagement_v3_eventTypes-~\_patch_1m | 200 | 200 | 200 | 200 | 200 | | core_eventmanagement_v3_eventTypes-~\_patch_1h | 1000 | 1000 | 1000 | 1000 | 1000 | | core_eventmanagement_v3_eventTypes-~\_get_1m | 500 | 500 | 500 | 500 | 500 | | core_eventmanagement_v3_eventTypes-~\_get_1h | 6000 | 6000 | 6000 | 6000 | 6000 | | core_eventmanagement_v3_eventTypes-~\_delete_1m | 200 | 200 | 200 | 200 | 200 | | core_eventmanagement_v3_eventTypes-~\_delete_1h | 1000 | 1000 | 1000 | 1000 | 1000 | | core_eventmanagement_v3_deleteEventsJobs_post_1m | 200 | 200 | 200 | 200 | 200 | | core_eventmanagement_v3_deleteEventsJobs_post_1h | 1000 | 1000 | 1000 | 1000 | 1000 | | core_eventmanagement_v3_deleteEventsJobs-~\_get_1m | 500 | 500 | 500 | 500 | 500 | | core_eventmanagement_v3_deleteEventsJobs-~\_get_1h | 6000 | 6000 | 6000 | 6000 | 6000 | | core_eventmanagement_v3_createEventsJobs_post_1m | 200 | 200 | 200 | 200 | 200 | | core_eventmanagement_v3_createEventsJobs_post_1h | 1000 | 1000 | 1000 | 1000 | 1000 | | core_eventmanagement_v3_createEventsJobs-~\_get_1m | 500 | 500 | 500 | 500 | 500 | | core_eventmanagement_v3_createEventsJobs-~\_get_1h | 6000 | 6000 | 6000 | 6000 | 6000 | Following Event Object limits are applied for P&P2 public cloud customers. - Event: 7500000 - EventType: 300 Note Rate Limits can cause discontinuity in the use case flow. Ensure to check the rate limits before implementing APIs.There are similar limits for "pay as you go" customers. # Event Management - Async API Specification [Download Async API Specification](/insights-hub/api_specs/eventmanagement-async-v3-0-1.yaml) # Event Management Async API Service – API Overview ## Available API versions | Region | API version | | -------- | ------------------------------------------------- | | Europe 1 | [3.1.0](api-eventmanagement-async-api-3-0-1.html) | # Event Management – Best Practices This document provides best practices for querying and creating events or event types in Event Management. ## General Remarks - The size of a page should be as small as possible (default page size is 20). - If real time data about events is not required, set the polling cycle as high as possible. - If an application uses retry strategy, it should wait for some time after receiving a gateway timeout. - For the following severities default apps like Insights Hub Monitor or the Insights Hub Web Components show the depending icon. For other severities only the number is displayed. | Severity | Icon | Description | | -------- | ---- | ----------- | | 20 | | Error | | 30 | | Warning | | 40 | | Information | ## Filterable Properties for Custom Events ### Enum If a custom event type field stores only allows a few values, consider using ENUM as type of that field. ```json { "name": "volatileEventType", "ttl": 1, "fields": [ { "name": "risk", "filterable": true, "required": true, "updatable": true, "type": "ENUM", "values" : ["LOW", "MEDIUM", "HIGH"] } ] } ``` ### UUID vs String If a custom event type field stores only UUID values, consider using UUID as type of that field instead of STRING. ```json { "name": "volatileEventType2", "ttl": 1, "fields": [ { "name": "code", "filterable": true, "required": true, "updatable": false, "type": "UUID" } ] } ``` ## Filtering `HTTP GET /events` API uses `filter` parameter to filter custom events list. Please refer to [Listing Custom Events](api-eventmanagement-samples-events.html#listing-custom-events) for sample. ### Filtering by Timestamp When filtering by timestamp the time interval should be as small as possible. For better performance of visualizations, it should not exceed 1 month. If no `timestamp` is specified for the filter, only events from the last week are retrieved. ```json { "timestamp": { "between": "[1970-01-01T07:25:07.166Z,2018-11-05T10:25:07.166Z)" } } ``` ```json { "timestamp": { "between": "[2018-11-01T07:25:07.166Z,2018-11-05T10:25:07.166Z)" } } ``` ### Filtering by Timestamp (Using Between instead of Before/After) When filtering by timestamp, the time interval should be as small as possible. For better performance of visualizations, it should not exceed 1 month. If `Before` & `After` is used, it will be any range in past & future respectively. It should be used judicially when you are aware about these factors mostly in non-performant operations like KPI calculation/scheduled operations. If no `timestamp` is specified for the filter, only events from the last week are retrieved. ```json { "timestamp": { "before": "2023-11-05T10:25:07.166Z" } } ``` ```json { "timestamp": { "between": "[2023-11-01T07:25:07.166Z,2023-11-05T10:25:07.166Z)" } } ``` ### Filtering by Type ID Provide the `typeId` for the filter to search among events of a particular event type. ```json { "timestamp": { "after": "2018-11-01T07:25:07.166Z" } } ``` ```json { "typeId": "3868feab-9ae6-44a8-9d9e-d20da848afb8", "timestamp": { "after": "2018-11-01T07:25:07.166Z" } } ``` ### Filtering by other Fields Filter by custom fields rather than standard ones, if possible. ```json { "timestamp":{ "between":"[2018-10-01T00:00:00.001Z,2018-10-05T14:53:07.534Z)" }, "entityId":"e7a12da7b60c4402a0a14dce547206ed", "acknowledged":false, "typeId":"com.siemens.mindsphere.eventmgmt.event.type.MindSphereStandardEvent" } ``` ```json { "timestamp":{ "between":"[2018-10-01T00:00:00.001Z,2018-10-05T14:53:07.534Z)" }, "entityId":"e7a12da7b60c4402a0a14dce547206ed", "acknowledged":false, "typeId":"3f690adf-edc8-447f-b1c8-4758aa7924ba", "risk": "LOW" } ``` ### Drilldown Sometimes it is faster to do a drilldown, i.e. the filtering is done in multiple steps. For example, the filter expression below filters for a `typeId` which refers to a custom event type. However, the other filter parameters refer to fields defined in the parent event type. This can make the filtering more expensive.\ In such cases it is advised to first replace the `typeId` by the parent type. Afterwards a drilldown to the for filtering by fields of the custom event type can be done. ```json { "timestamp":{ "between":"[2018-10-01T00:00:00.001Z,2018-10-05T14:53:07.534Z)" }, "entityId":"e7a12da7b60c4402a0a14dce547206ed", "acknowledged":false, "typeId":"b7f9a843-a530-4159-989e-20645e2b647d" } ``` ```json { "timestamp":{ "between":"[2018-10-01T00:00:00.001Z,2018-10-05T14:53:07.534Z)" }, "entityId":"e7a12da7b60c4402a0a14dce547206ed", "acknowledged":false, "typeId":"com.siemens.mindsphere.eventmgmt.event.type.MindSphereStandardEvent" } ``` ### Filtering with PageCache For better performance, set `enablePageCache` to `true` along with the filter while retrieving the events. When there are huge number of events satisfying the filter criteria, many subsequent calls are needed for traversing all the pages and retrieving the events. For such cases, we have introduced a new query parameter `enablePageCache` which can be used as below: - Set `enablePageCache` to `false` for first request. For all subsequent request for different pages, set `enablePageCache` to `true`. - Set `enablePageCache` to `true` for all requests if the new events are getting generated less frequently, i.e. having intervals of more than 10 mins. When this parameter is set to `true`, total count of events satisfying the filter criteria will be cached for 10 mins and same cached value will be returned in response resulting in near to accurate value. The event information returned will always be the latest. ```json { enablePageCache : false } ``` ```json { enablePageCache : true } ``` ## Sorting For better performance, sort by fields that are present in the filter expression. ```json { "typeId":"com.siemens.mindsphere.eventmgmt.event.type.MindSphereStandardEvent" } ``` ```json { "timestamp":{ "between":"[2018-10-01T00:00:00.001Z,2018-11-01T14:53:07.534Z)" }, "typeId":"com.siemens.mindsphere.eventmgmt.event.type.MindSphereStandardEvent" } ``` ```json sort: timestamp,desc ``` 'timestamp' field may contain duplicate values. For better performance with pagination add unique value fields in sort parameter like 'id'. ```json sort: timestamp,id,desc ``` # Event Count Estimator (Event Management) ## Idea The Event Count Estimator helps to interactively calculate the approximate number of events generated by all assets in the system for a particular time period. Disclaimer The Event count from the estimator may vary depending on the number of events generated by asset and TTL for events. It estimates the event count purely based on user input. ## Basics - Number of Assets: These are digital Assets modelled in the system. - Average number of events per day per asset: This is average number of events produced by a single asset per day. - TTL: This is Time to Live for events. Expected Retention time once an event is generated (in days). Minimum ttl is 1-day, maximum ttl is 1825 day (5 yrs). Default value of ttl is 35 days. After the TTL is over, the events are removed from the system freeing the events quota. ## Estimation of Event Count - Event count is calculated based on the total number of assets in the system. - The user needs to add the approximate number of events each asset will generate. Ad-hoc events are not considered here. - Based on the above two inputs, total events per day will be calculated as: number of assets * number of events per asset. - Then the user needs to specify the TTL for events. This can be between 1 to 1825 days. Removing the events from the system once the TTL is over is the important parameter in event count calculation. - The user can also select the number of months for which event estimation is required (maximum 36 months). - Event count estimation result is shown in graph and list view. The graph is plotted against the number of events per month and count. Based on event estimation, the user can decide which event quota plan it needs to purchase. ## Event Count Estimator | Paramaters | Number | | | ----------------------------------------------------------------------- | ------ | --- | | Assets (These are digital Assets modelled in Insights Hub) | | | | Number of Events expected per Asset per day | | | | Expected Retention time once an event is generated (in days) - max 1825 | | | | Total Events generated per day | | | | Duration for estimation in months | | | \[Calculate Events\](javascript:estimateEventsList() ) Graph List | Months | Number Of events | | ------ | ---------------- | | Plan | Event Quota | | ------- | ----------- | | Plan XS | 50000 | | Plan S | 150000 | | Plan M | 500000 | | Plan L | 1250000 | | Plan XL | 2500000 | **(The color of box corresponds with the color of the graph shows same information about the plan.)** # Event Management Advance The owner filter functions within the Get ALL eventTypes API will support 'equality' and 'IN' operators from the upcoming release. ## Idea There are many devices that are connected to Insights Hub. Sometimes, devices show an unexpected behavior (for example, leakage in a motor pump or freezer exceeding the temperature limit) which demands special attention. These situations are known as events which can be digitally represented in Insights Hub. To capture these generated events, we have the Event Management Service that allows users to create, update, and delete events associated with the configured Assets. The service offers flexibility in modeling events, allowing users to define event types and include custom fields to capture all the necessary details according to their specific use case. Additionally, users can acknowledge an event once the necessary corrective action has been taken on the device. EventType is metadata used to create events in system. Events are associated with assets/entity. Users can use predefined event types or create their own templates for generating new events. The user is able to filter, update or delete the events or event types. Note Devices is nothing but entityid or assetid. Consider the given scenario to understand Event Management Service.\ A freezer is required to maintain a temperature of approximately 36 degrees to preserve the freshness of stored food. If the temperature exceeds this limit, there is a risk of food spoilage. To address this concern and protect our food, an event needs to be generated when the freezer temperature rises above 36 degrees. This event can be captured using the Event Management Service, which allows for the creation and management of events associated with devices. The service enables users to define custom event types with relevant details, facilitating appropriate actions to rectify the situation and ensure the safety of stored food. ## Access The Event Management is exposed to developers as a REST API. For accessing this service, you need to have the respective roles listed in [Event Management roles and scopes](https://documentation.mindsphere.io/MindSphere/concepts/concept-roles-scopes.html#event-management). For accessing the [Secure Data Sharing (SDS)](../core-resourceaccessmanagement/api-resourceaccessmanagement-services-actions-map.html#policy-based-access-control) protected APIs, you need to have appropriate Policy Definitions in place. For the list of supported APIs and required actions, refer to [Required Policy actions by Industrial IoT services](../core-resourceaccessmanagement/api-resourceaccessmanagement-services-actions-map.html#event-management). ## Basics Users can use predefined event types or create their own templates for generating new events. The user is able to filter, update or delete the events or event types. Info For advice on how to use this service, refer to the [Best Practices](api-eventmanagement-best-practices.html). ### Events There are two kinds of events: standard events and custom events. Both inherit the following fields from a common ancestor: - id - correlationId - timestamp (required) - entityId (required) The following additional fields are available: - **Standard Events**\ A standard event is created from a predefined event type, for which the following base properties are available: - description - severity - code - source - acknowledged - **Custom Events**\ A custom event is created from a user-defined event type. The available properties are defined in the event type. ### Event types EventType is metadata used to create events in system. Events are associated with assets/entity. There are two kinds of **event types**: 1. **Standard Event Type**\ It is a predefined event type in the system with a time-to-live (TTL) of 35 days. 1. **Custom Event Type**\ It is the event type which is defined by the user and created by them in system.\ Events created using custom event types are called custom events. These define a set of custom fields which hold the properties of the custom event.\ Each custom field of an event type has a data type and can be defined to be required and/or filterable.\ Users can use standard event types or create their own custom event types for generating new events. The user is able to filter, update or delete the events or event types. An event type has a global or local scope and a time to live (TTL in days). TTL is applied to all instantiated events and expiry time of event is calculated at time of creation. Event types can be derived from other (parent) event types. An inheritance hierarchy of up to 4 levels is allowed. Every event type inherits all fields from its ancestor event types. Event types of other tenants with scope as global would only be seen if the package or application of those tenants have been purchased or subscribed, or they are a part of PNP capability packs. Event Types can also be shared via tenant collaboration ### Filtering and sorting Events can be filtered and sorted by all assigned base properties except `description`. Additionally, if the event type is set as filter parameter, events can be filtered and sorted by all custom fields specified as filterable. Events can also be filtered and sorted by all inherited custom fields. The filter syntax can be customized using operators like `and`, `or`, `not equals`, `contains` or `startsWith`. Be aware that only a single, not negated event type can be used as filter parameter. ### Updating Events can be updated. There are two ways to update an event: A PUT request overwrites an event and a POST request creates a history. This means the original event remains in the database and a new event is created with the same `entityId` and `correlationId` but a different id. Listing events returns only the latest of an event's history. The whole history of the event is listed with `history` query parameter set to true. ### Deleting Events can be deleted based on a custom filter asynchronously via POST request. If the latest event instance in a history of an event is deleted then the history of the event will be deleted. ## Features The Event Management Service exposes its API for realizing the following tasks: **Event Operations:** - Create an event. - Get a single event. - List filtered and sorted events. - Update an existing event (PUT). - Create a historic event while updating an event (POST). - Delete events asynchronously based on custom filter. - Create events asynchronously. - Sharing of the events between collaborating tenant - Get notification on event create/delete/update **EventType Operations:** - Create, update, delete an event type - Get a single event type. - List filtered and sorted event types. - Sharing of the EventType between collaborating tenants **Sharing:** All event Types are shared between collaborating tenants. Events can be shared by asset sharing between collaborating tenants. ## Limitations - All limitations and restrictions are documented in the API specification. For details, refer to the definitions for: - request parameters - query parameters - error responses - If there are too many API requests, the service throttles them temporarily with a 429 status code. A Retry-After header might be included with this response, indicating how long in seconds to wait before making a new request. - [API rate limits](api-eventmanagement-apiratelimits.html) for EventManagement are applicable as technical rate limits. - Custom Event type creation will have following restrictions: - Maximum number of custom 'fields' should not be more than 100. - Total STRING fields in custom eventtype should not be more than 40. - Total LINK fields in custom eventtype should not be more than 1. - Total ENUM fields in custom eventtype should not be more than 3. - ENUM constants per ENUM type should not be more than 50 - Field name should always start with lowercase letter. For example, if we want field name as temperature then "Temperature" is invalid name, correct name is "temperature". - By default, legacy value plan tenant can create maximum of 2.5 million events. Beyond that more events can be purchased via Upgrade. - Currently, Child eventtype LOCAL scope can be updated to GLOBAL having LOCAL scope for parent eventtype. - Eventtype creation is allowed even though values definition is empty in case of ENUM field type. ## Example Scenario Suppose a Brewery owner is using thermometer to measure the temperature while making beer and the temperature must not cross the defined temperature limit, let's assume it as 60 degree Celsius. Brewery owner wants to generate event as soon as the temperature increases above 55 degree Celsius. Brewery owner has to follow below steps for that 1. Add the thermometer as an asset in Insights Hub. Let's assume assetId/EntityId is 'abcd'. 1. Create an EventType for the same. Let's assume EventType is 'pqrs'. ```json { "id": "pqrs", "name": "temperatureAlert", "parentId": "parentEventTypeId", "ttl": 3, "scope": "LOCAL", "fields": [ { "name": "temperature", "filterable": true, "required": true, "updatable": false, "type": "integer", } ] } ``` 1. Brewery owner has to collect the Data using timeseries and create Rule to generate events for temperature value greater than 55 degree Celsius. 1. After this events will be automatically created when the temperature increases above 55 degree Celsius. ## Related Links - [Best Practices](api-eventmanagement-best-practices.html) - [Asset Management](../advanced-assetmanagement/api-assetmanagement-overview.html) # Event Management Service – Filtering When reading events (using GET All /events api), all events are filtered for a tenant whose token is used for making a request. In addition, you can supply one or more of the following field names in the filter queries and any of your custom field which has filterable property set to true : - `entityId` - `typeId` - `correlationId` - `timestamp` When reading events of Standard Type or any of its custom child type, you can use below additional fields in the filter query : - `acknowledged` - `source` - `severity` - `code` Wildcards are not supported. Timestamps must be in the following format: `2018-01-01T00:00:00.00Z` When a filter has no matches, the response will be empty. ## Filter functions ### Function `eq` (equals) ```json { "name": "MyMotor" } ``` or ```json { "name": { "eq": "MyMotor" } } ``` ### Function `in` The function `in` is very similar to `equals`, but instead matching to one value, it checks an array of values, and matches for any of them. ```json { "name": { "in": { "value": ["MyMotor", "MyEngine"] } } } ``` ### Function `endsWith` ```json { "name": { "endsWith": "Motor" } } ``` ### Function `startsWith` ```json { "name": { "startsWith": "My" } } ``` ### Function `before` ```json { "timestamp": { "before": "2018-01-01T00:00:00.00Z" } } ``` ### Function `after` ```json { "timestamp": { "after": "2018-01-01T00:00:00.00Z" } } ``` ### Function `between` ```json { "timestamp": { "between": "[2018-01-01T00:00:00.00Z, 2018-01-31T00:00:00.00Z)" } } ``` ## Filter operations ### Operation `or` ```json { "name": { "or": [{ "eq": "MyMotor" }, { "endsWith": "Motor" }] } } ``` or ```json { "or": { "deleted": { "eq": null }, "name": { "startsWith": "My" } } } ``` ### Operation `and` ```json { "name": "MyMotor", "deleted": null } ``` or ```json { "and": { "deleted": { "eq": null }, "name": { "startsWith": "My" } } } ``` # Event Management – Event Type Operations ## Creating a Custom Event Type Create an event type with `temperature` field, derived from the event type `385aafd8-c919-11e7-abc4-cec278b6b50a`: `HTTP POST /eventTypes` ```json { "name" : "MyMotorEventType", "ttl" : 500, "scope" : "LOCAL", "parentId" : "385aafd8-c919-11e7-abc4-cec278b6b50a", "fields" : [ { "name" : "temperature", "filterable" : false, "required" : false, "updatable" : true, "type" : "STRING" } ] } ``` To provide custom id for eventype use format : \.\.\ Note \ is name of tenant, can be obtained using TenantManagements /tenantInfo API [TenantManagement](https://developer.siemens.com/insights-hub/docs/apis/core-tenantmanagement/api-tenantmanagement-api.html). The response shows the content and link to the created event type: ```json { "id" : "c3d6749a-c9e4-11e7-abc4-cec278b6b50a", "name" : "MyMotorEventType", "ttl" : 500, "etag" : 0, "owner" : "phoenix-tenant", "scope" : "LOCAL", "parentId" : "385aafd8-c919-11e7-abc4-cec278b6b50a", "fields" : [ { "name" : "temperature", "filterable" : false, "required" : false, "updatable" : true, "type" : "STRING" } ], "_links" : { "self" : { "href" : "https://eventmanagement.advanced.mindsphere.io/eventTypes/c3d6749a-c9e4-11e7-abc4-cec278b6b50a" }, "events" : { "href" : "https://eventmanagement.advanced.mindsphere.io/events?filter=%7B%22typeId%22:%22c3d6749a-c9e4-11e7-abc4-cec278b6b50a%22%7D" } } } ``` ## Reading a Custom Event Type Read an event type by id: `HTTP GET /eventTypes/c3d6749a-c9e4-11e7-abc4-cec278b6b50a` The response shows the content and link to the requested event type: ```json { "id" : "c3d6749a-c9e4-11e7-abc4-cec278b6b50a", "name" : "MyMotorEventType", "ttl" : 500, "etag" : 0, "owner" : "phoenix-tenant", "scope" : "LOCAL", "parentId" : "385aafd8-c919-11e7-abc4-cec278b6b50a", "fields" : [ { "name" : "temperature", "filterable" : false, "required" : false, "updatable" : true, "type" : "STRING" } ], "_links" : { "self" : { "href" : "https://eventmanagement.advanced.mindsphere.io/eventTypes/c3d6749a-c9e4-11e7-abc4-cec278b6b50a" }, "events" : { "href" : "https://eventmanagement.advanced.mindsphere.io/events?filter=%7B%22typeId%22:%22c3d6749a-c9e4-11e7-abc4-cec278b6b50a%22%7D" } } } ``` ## Listing Custom Event Types Get a list of all event types sorted in `alphabetically` `ascending` order of their `name` field: `HTTP GET /eventTypes?page=0&size=20&sort=name%2Casc` The response shows the list of event types according to provided filter information: ```json { "_embedded" : { "eventTypes" : [ { "id" : "x9x9949a-c9e4-11e7-abc4-cec278b6x99x", "name" : "ACompletelyDifferentMotorEventType", "ttl" : 500, "etag" : 0, "owner" : "phoenix-tenant", "scope" : "LOCAL", "parentId" : "385aafd8-c919-11e7-abc4-cec278b6b50a", "fields" : [ { "name" : "pressure", "filterable" : false, "required" : false, "updatable" : true, "type" : "STRING" } ], "_links" : { "self" : { "href" : "https://eventmanagement.advanced.mindsphere.io/eventTypes/c3d6749a-c9e4-11e7-abc4-cec278b6b50a" }, "events" : { "href" : "https://eventmanagement.advanced.mindsphere.io/events?filter=%7B%22typeId%22:%22c3d6749a-c9e4-11e7-abc4-cec278b6b50a%22%7D" } } }, { "id" : "c3d6749a-c9e4-11e7-abc4-cec278b6b50a", "name" : "MyMotorEventType", "ttl" : 500, "etag" : 0, "owner" : "phoenix-tenant", "scope" : "LOCAL", "parentId" : "385aafd8-c919-11e7-abc4-cec278b6b50a", "fields" : [ { "name" : "temperature", "filterable" : false, "required" : false, "updatable" : true, "type" : "STRING" } ], "_links" : { "self" : { "href" : "https://eventmanagement.advanced.mindsphere.io/eventTypes/x9x9949a-c9e4-11e7-abc4-cec278b6x99x" }, "events" : { "href" : "https://eventmanagement.advanced.mindsphere.io/events?filter=%7B%22typeId%22:%22x9x9949a-c9e4-11e7-abc4-cec278b6x99x%22%7D" } } } ] }, "_links" : { "self" : { "href" : "https://eventmanagement.advanced.mindsphere.io/eventTypes{?filter}", "templated" : true } }, "page" : { "size" : 20, "totalElements" : 2, "totalPages" : 1, "number" : 0 } } ``` ## Updating a Custom Event Type Add a new field to an existing event type: `HTTP PATCH /eventTypes/c3d6749a-c9e4-11e7-abc4-cec278b6b50a` ```json { "op" : "add", "path" : "/fields", "value" : { "name" : "newField", "updatable" : false, "type" : "INTEGER" } } ``` The response shows the content and link to the updated event type: ```json { "id" : "c3d6749a-c9e4-11e7-abc4-cec278b6b50a", "name" : "MyMotorEventType", "ttl" : 500, "etag" : 1, "owner" : "phoenix-tenant", "scope" : "LOCAL", "parentId" : "385aafd8-c919-11e7-abc4-cec278b6b50a", "fields" : [ { "name" : "originalField", "filterable" : false, "required" : true, "updatable" : false, "type" : "STRING" }, { "name" : "newField", "filterable" : false, "required" : false, "updatable" : false, "type" : "INTEGER" } ], "_links" : { "self" : { "href" : "https://eventmanagement.advanced.mindsphere.io/eventTypes/c3d6749a-c9e4-11e7-abc4-cec278b6b50a" }, "events" : { "href" : "https://eventmanagement.advanced.mindsphere.io/events?filter=%7B%22typeId%22:%22c3d6749a-c9e4-11e7-abc4-cec278b6b50a%22%7D" } } } ``` Modify an existing field of an existing event type: `HTTP PATCH /eventTypes/c3d6749a-c9e4-11e7-abc4-cec278b6b50a` ```json { "op" : "replace", "path" : "/fields/originalField/required", "value" : false } ``` The response shows the content and link to the updated event type: ```json { "id" : "c3d6749a-c9e4-11e7-abc4-cec278b6b50a", "name" : "MyMotorEventType", "ttl" : 500, "etag" : 1, "owner" : "phoenix-tenant", "scope" : "LOCAL", "parentId" : "385aafd8-c919-11e7-abc4-cec278b6b50a", "fields" : [ { "name" : "originalField", "filterable" : false, "required" : false, "updatable" : true, "type" : "INTEGER" } ], "_links" : { "self" : { "href" : "https://eventmanagement.advanced.mindsphere.io/eventTypes/c3d6749a-c9e4-11e7-abc4-cec278b6b50a" }, "events" : { "href" : "https://eventmanagement.advanced.mindsphere.io/events?filter=%7B%22typeId%22:%22c3d6749a-c9e4-11e7-abc4-cec278b6b50a%22%7D" } } } ``` ## Deleting a Custom Event Type Delete an existing event type by id: `HTTP DELETE /eventTypes/c3d6749a-c9e4-11e7-abc4-cec278b6b50a` No content is returned. ## Sharing of Event Type Event Type with "LOCAL" scope is by default available to tenants who had established collaborations between them. Param "includeShared" is needed to query shared Event Types. `HTTP GET /eventTypes?includeShared=true` # Event Management – Event Operations ## Creating a Standard Event Create a standard event: `HTTP POST /events` ```json { "correlationId" : "cd5f73c4-bd67-11e7-abc4-cec278b6b50a", "timestamp" : "2018-07-29T06:09:29.743Z", "entityId" : "cd5f73c4-bd67-11e7-abc4-cec278b6b50b", "code" : "123", "description" : "Error happened in the engine", "severity" : 5, "source" : "eventSource", "acknowledged" : false } ``` The response shows the content and link to the created event: ```json { "id" : "ab5f73c4-bd67-11e7-abc4-cec278b6b50a", "typeId" : "com.siemens.mindsphere.eventmgmt.event.type.MindSphereStandardEvent", "correlationId" : "cd5f73c4-bd67-11e7-abc4-cec278b6b50a", "timestamp" : "2018-07-29T06:09:29.743Z", "entityId" : "cd5f73c4-bd67-11e7-abc4-cec278b6b50b", "etag" : 1, "_links" : { "self" : { "href" : "https://eventmanagement.advanced.mindsphere.io/events/ab5f73c4-bd67-11e7-abc4-cec278b6b50a" } }, "severity" : 5, "code" : "123", "acknowledged" : false, "description" : "Error happened in the engine", "source" : "eventSource" } ``` ## Creating a Custom Event Create a custom event: - All the required fields from custom EventType and its parent eventTypes in the hierarchy should be included in the event payload when creating a custom event. - Suppose that a child EventType with the id 'c3d6749a-c9e4-11e7-abc4-cec278b6b50a' is created from a parent EventType that has a required field as 'additionalFieldParent'. The 'additionalFieldParent' must be included in the event payload when creating an event for a child EventType. `HTTP POST /events` ```json { "correlationId" : "cd5f73c4-bd67-11e7-abc4-cec278b6b50a", "timestamp" : "2018-07-29T06:09:29.743Z", "entityId" : "cd5f73c4-bd67-11e7-abc4-cec278b6b50b", "typeId" : "c3d6749a-c9e4-11e7-abc4-cec278b6b50a", "additionalFieldChild" : "additionalFieldChildValue", "additionalFieldParent" : "additionalFieldParentalue" } ``` The response shows the content and link to the created event: ```json { "id" : "ab5f73c4-bd67-11e7-abc4-cec278b6b50a", "typeId" : "c3d6749a-c9e4-11e7-abc4-cec278b6b50a", "correlationId" : "cd5f73c4-bd67-11e7-abc4-cec278b6b50a", "timestamp" : "2018-07-29T06:09:29.743Z", "entityId" : "cd5f73c4-bd67-11e7-abc4-cec278b6b50b", "etag" : 1, "_links" : { "self" : { "href" : "https://eventmanagement.advanced.mindsphere.io/events/ab5f73c4-bd67-11e7-abc4-cec278b6b50a" } }, "additionalFieldChild" : "additionalFieldChildValue", "additionalFieldParent" : "additionalFieldParentalue" } ``` ## Creating Multiple Events Run a job which creates two events: `HTTP POST /createEventsJobs` ```json { "events" : [ { "correlationId" : "cd5f73c4-bd67-11e7-abc4-cec278b6b50a", "timestamp" : "2018-07-29T12:17:48.454Z", "entityId" : "cd5f73c4-bd67-11e7-abc4-cec278b6b50b", "typeId" : "c3d6749a-c9e4-11e7-abc4-cec278b6b50a" }, { "correlationId" : "97b33fb8-c536-11e7-abc4-cec278b6b99x", "timestamp" : "2018-07-29T12:17:48.454Z", "entityId" : "pp9973c4-bd67-11e7-abc4-cec278b6po77", "typeId" : "x9x9949a-c9e4-11e7-abc4-cec278b6x99x" } ] } ``` The response shows the ID and the state of the request: ```json { "id" : "cd5f73c4-bd67-11e7-abc4-cec278b6b50b", "state" : "ACCEPTED" } ``` The state of the job created above can be checked by the following request: `HTTP GET /createEventsJobs/cd5f73c4-bd67-11e7-abc4-cec278b6b50b` The response shows the current state of the job: ```json { "id" : "cd5f73c4-bd67-11e7-abc4-cec278b6b50b", "state" : "FINISHED_WITH_ERROR", "details" : { "resultDescription" : [ { "event" : { "correlationId" : "cd5f73c4-bd67-11e7-abc4-cec278b6b50a", "timestamp" : "2018-07-29T12:17:48.454Z", "typeId" : "c3d6749a-c9e4-11e7-abc4-cec278b6b50a", "customStringField" : "customValue" }, "resultCode" : "400", "errorMessage" : "Missing entity id" }, { "event" : { "customStringField" : "CustomStringValue", "_links" : { "self" : { "href" : "https://eventmanagement.advanced.mindsphere.io/events/ab5f73c4-bd67-11e7-abc4-cec278b6b50a" } }, "typeId" : "c3d6749a-c9e4-11e7-abc4-cec278b6b50a", "correlationId" : "cd5f73c4-bd67-11e7-abc4-cec278b6b50a", "entityId" : "cd5f73c4-bd67-11e7-abc4-cec278b6b50b", "etag" : 0, "id" : "ab5f73c4-bd67-11e7-abc4-cec278b6b50a", "timestamp" : "2018-07-29T12:17:48.454Z" }, "resultCode" : "201" } ] } } ``` ## Reading a Standard Event Request an event by ID: `HTTP GET /events/ab5f73c4-bd67-11e7-abc4-cec278b6b50a` The response shows the content and link to the requested event: ```json { "id" : "ab5f73c4-bd67-11e7-abc4-cec278b6b50a", "typeId" : "com.siemens.mindsphere.eventmgmt.event.type.MindSphereStandardEvent", "correlationId" : "cd5f73c4-bd67-11e7-abc4-cec278b6b50a", "timestamp" : "2018-07-29T06:09:29.743Z", "entityId" : "cd5f73c4-bd67-11e7-abc4-cec278b6b50b", "etag" : 1, "_links" : { "self" : { "href" : "https://eventmanagement.advanced.mindsphere.io/events/ab5f73c4-bd67-11e7-abc4-cec278b6b50a" } }, "severity" : 5, "code" : "123", "acknowledged" : false, "description" : "Error happened in the engine", "source" : "eventSource" } ``` ## Reading a Custom Event Request an event by ID: `HTTP GET /events/ab5f73c4-bd67-11e7-abc4-cec278b6b50a` The response shows the content and link to the requested event: ```json { "id" : "ab5f73c4-bd67-11e7-abc4-cec278b6b50a", "typeId" : "c3d6749a-c9e4-11e7-abc4-cec278b6b50a", "correlationId" : "cd5f73c4-bd67-11e7-abc4-cec278b6b50a", "timestamp" : "2018-07-29T06:09:29.743Z", "entityId" : "cd5f73c4-bd67-11e7-abc4-cec278b6b50b", "etag" : 1, "_links" : { "self" : { "href" : "https://eventmanagement.advanced.mindsphere.io/events/ab5f73c4-bd67-11e7-abc4-cec278b6b50a" } }, "additionalField" : "additionalFieldValue" } ``` ## Listing Standard Events List standard events sorted in `descending` order by the `timestamp` of their `history` field: `HTTP GET /events?page=0&size=20&sort=timestamp%2Cdesc&history=true` The response shows the list of events according to provided filter information: ```json { "_embedded" : { "events" : [ { "id" : "ab5f73c4-bd67-11e7-abc4-cec278b6b50a", "typeId" : "com.siemens.mindsphere.eventmgmt.event.type.MindSphereStandardEvent", "correlationId" : "cd5f73c4-bd67-11e7-abc4-cec278b6b50a", "timestamp" : "2018-07-29T06:09:29.743Z", "entityId" : "cd5f73c4-bd67-11e7-abc4-cec278b6b50b", "etag" : 1, "_links" : { "self" : { "href" : "https://eventmanagement.advanced.mindsphere.io/events/ab5f73c4-bd67-11e7-abc4-cec278b6b50a" } }, "severity" : 5, "code" : "123", "acknowledged" : false, "description" : "Error happened in the engine", "source" : "eventSource" }, { "id" : "xy5f73c4-bd67-11e7-abc4-cec278b6b50a", "typeId" : "com.siemens.mindsphere.eventmgmt.event.type.MindSphereStandardEvent", "correlationId" : "cd5f73c4-bd67-11e7-abc4-cec278b6b50a", "timestamp" : "2018-07-28T06:09:29.745Z", "entityId" : "cd5f73c4-bd67-11e7-abc4-cec278b6b50b", "etag" : 1, "_links" : { "self" : { "href" : "https://eventmanagement.advanced.mindsphere.io/events/xy5f73c4-bd67-11e7-abc4-cec278b6b50a" } }, "severity" : 3, "code" : "123", "acknowledged" : false, "description" : "Error happened in the engine", "source" : "eventSource" } ] }, "_links" : { "self" : { "href" : "https://eventmanagement.advanced.mindsphere.io/events?history=true{&filter}", "templated" : true } }, "page" : { "size" : 20, "totalElements" : 2, "totalPages" : 1, "number" : 0 } } ``` ## Listing Custom Events List custom events sorted in `descending` order by the `timestamp` of their `history` field: `HTTP GET /events?filter=%7B%22typeId%22%3A%22dud6749a-c9e4-11e7-abc4-cec278b6b50a%22%7D&page=0&size=20&sort=timestamp%2Cdesc&history=true` The response shows the list of events according to provided filter information: ```json { "_embedded" : { "events" : [ { "id" : "ab5f73c4-bd67-11e7-abc4-cec278b6b50a", "typeId" : "dud6749a-c9e4-11e7-abc4-cec278b6b50a", "correlationId" : "cd5f73c4-bd67-11e7-abc4-cec278b6b50a", "timestamp" : "2018-07-29T06:09:29.743Z", "entityId" : "cd5f73c4-bd67-11e7-abc4-cec278b6b50b", "etag" : 1, "_links" : { "self" : { "href" : "https://eventmanagement.advanced.mindsphere.io/events/ab5f73c4-bd67-11e7-abc4-cec278b6b50a" } }, "additionalField" : "additionalFieldValue" }, { "id" : "zu5f73c4-bd67-11e7-abc4-cec278b6b50a", "typeId" : "dud6749a-c9e4-11e7-abc4-cec278b6b50a", "correlationId" : "cd5f73c4-bd67-11e7-abc4-cec278b6b50a", "timestamp" : "2018-07-28T06:09:29.745Z", "entityId" : "cd5f73c4-bd67-11e7-abc4-cec278b6b50b", "etag" : 2, "_links" : { "self" : { "href" : "https://eventmanagement.advanced.mindsphere.io/events/zu5f73c4-bd67-11e7-abc4-cec278b6b50a" } }, "additionalField" : "additionalFieldValue" } ] }, "_links" : { "self" : { "href" : "https://eventmanagement.advanced.mindsphere.io/events?filter=%7B%22typeId%22:%22dud6749a-c9e4-11e7-abc4-cec278b6b50a%22%7D&history=true" } }, "page" : { "size" : 20, "totalElements" : 2, "totalPages" : 1, "number" : 0 } } ``` ## Updating an Event Update an event by ID: `HTTP PUT /events/ab5f73c4-bd67-11e7-abc4-cec278b6b50a` ```json { "correlationId" : "cd5f73c4-bd67-11e7-abc4-cec278b6b50a", "timestamp" : "2018-07-29T06:09:29.743Z", "entityId" : "ef5f73c4-bd67-11e7-abc4-cec278b6b50b", "description" : "Error happened in the engine" } ``` The response shows the content and link to the updated event: ```json { "id" : "ab5f73c4-bd67-11e7-abc4-cec278b6b50a", "typeId" : "com.siemens.mindsphere.eventmgmt.event.type.MindSphereStandardEvent", "correlationId" : "cd5f73c4-bd67-11e7-abc4-cec278b6b50a", "timestamp" : "2018-07-29T06:09:29.743Z", "entityId" : "cd5f73c4-bd67-11e7-abc4-cec278b6b50b", "etag" : 2, "_links" : { "self" : { "href" : "https://eventmanagement.advanced.mindsphere.io/events/ab5f73c4-bd67-11e7-abc4-cec278b6b50a" } }, "severity" : 5, "code" : "123", "acknowledged" : false, "description" : "Error happened in the engine", "source" : "eventSource" } ``` ## Deleting Events Delete events by filter: `HTTP POST /deleteEventsJobs` ```json { "filter" : { "typeId" : "anyTypeId" } } ``` The response shows the ID and the state of the request: ```json { "id" : "cd5f73c4-bd67-11e7-abc4-cec278b6b50b", "state" : "ACCEPTED" } ``` The status of the job created above can be checked by the following request: `HTTP GET /deleteEventsJobs/cd5f73c4-bd67-11e7-abc4-cec278b6b50b` The response shows the current state of the job: ```json { "id" : "cd5f73c4-bd67-11e7-abc4-cec278b6b50b", "state" : "IN_PROGRESS" } ``` This response shows a successfully finished delete: ```json { "id": "a3a35ea8-5893-4aac-af98-8fac29148386", "state": "FINISHED", "details": { "resultCode": "200", "resultDescription": "1 events deleted." } } ``` ## Sharing of events Events can be shared between collaborating tenants. Events are associated with assets and events are shared implicitly if the corresponding asset is shared, i.e, a sharer tenant needs to share with the asset with receiver tenant to share the events with receiver tenant. The receiver tenant is only allowed to read the events. Param "includeShared" is needed to query shared events. `HTTP GET /events?includeShared=true` # Notification for Event Creation or Update ## Overview With subscription based notification, the 3rd party applications can now integrate subscription to topics related to event objects. Such applications would be then able to receive notifications when there's a change in status of event objects. Applications can respond with desired actions upon receiving a notification such as using 'GET events by ID API' to fetch more details about the event if required. This feature enables developers to build optimized interactions in the applications and make them performant. To create and publish App for consuming Event notification, refer to [How to Subscribe to notifications for event status change](https://documentation.mindsphere.io/MindSphere/howto/howto-receive-notifications.html) Note Subscription based notification is add-on offering on Public Cloud. ## Limitation - The time interval for publishing notification is 1 minute. - The underlying messaging service may bring nomenclature/version changes in near term; which may need changes and adaption in customer application. ## References - Message Broker API - Provisioning API ## FAQs 1. What is notification level in asset model? - The notification will be available at tenant level. 1. What is the maximum number of events information per notification? - The maximum number of Events information in one message will be 1000. 1. Can customer get delayed notification? - Yes, in case any Insights Hub component is non-operational. # Event Notifications ## Notification for event create or update - When events are created or updated in system, user will receive notification. - The user can expect maximum of 1000 events information in one notification message. - The notification message will contain information like id of the event, assetId and timestamp. The tenant and topicId will be provided as part of header for the message. Sample message notification when events are created or updated: ```json { "header": { "topic": "mdsp.core.eventmanagement.v1.pubsub.eventnotification", "tenantContext": [ "siemens" ] }, "events": [ { "id": "baeb542e-0730-46a9-88c9-d0982631645c", "assetId": "01096640c2164e84aa0442114147ea6e", "timestamp": "2021-07-12T03:55:00.000Z" }, { "id": "uaeo542e-0730-46a9-88c9-o0982631645c", "assetId": "01096640c2164e84aa0442114147ea6e", "timestamp": "2021-07-12T04:55:00.000Z" } ] } ``` # Notification Service – Samples for v4.x ## Mobile Push notifications ### Registering an application with Notification Service Notification Service provides the functionality for sending push notifications for an application installed on the respective mobile devices. Mobile applications must be registered in FCM (Android or iOS)/APNS (iOS) to receive push notifications. Either an iOS (sandbox & production) or Android configuration can be chosen for the app. Registration of the mobile app allows the developer to provide necessary push notification provider credentials for the app in Notification Service. App registration in Notification Service is a one-time activity for an app with respect to a tenant. Here, `fcmServerKey` is the unique server ID for an application used by FCM to send push notification. Instructions to provide the APNS related configurations for registering iOS app through this endpoint are given below. Execute the below command on the PEM file provided by APNS for an iOS app. This will provide `apnsSslCertificate` and an encrypted APNS app private key to `keys_out` text file. `openssl pkcs12 -in MyRootCA.p12 -out keys_out.txt` Execute the below command to extract `apnsAppPrivateKey` from the above generated text file i.e, `keys_out.txt` to `private.txt`. `openssl rsa -in keys_out.txt -out private.txt` `apnsSslCertificate` and `apnsAppPrivateKey` are used together for App registration in Notification Service. An application is registered using the following endpoint: ```http POST /mobileApps ``` Sample Request for Android mobile application: ```json { "name": "Voith_Service_App", "type": "android", "android": { "fcmServerKey": "serverKey" } } ``` Sample Request for iOS mobile application: ```json { "name": "Voith_Service_App", "type": "ios", "ios": { "bundleId": "com.your-company.app-name", "apnsSslCertificate": "ssl certificate in string format", "apnsAppPrivateKey": "private key in string format", "production": false } } ``` OR ```json { "name": "Voith_Service_App", "type": "ios", "ios": { "fcmServerKey": "serverKey" } } ``` Sample Response: ```json { "id": "59f92744-7bbc-4540-8b78-ee9976befc1a", "type": "ios", "name": "Voith_Service_App", "android": { "fcmServerKey": "serverKey" }, "ios": { "fcmServerKey": "serverKey", "bundleId": "com.your-company.app-name", "apnsSslCertificate": "ssl certificate in string format", "apnsAppPrivateKey": "private key in string format", "production": false } } ``` The ID generated `mobileAppId` is referenced further for triggering the notification. The following operations are also possible with a registered application. - Fetching all registered apps for a tenant - Updating a registered mobile app - Deleting an existing registered mobile app from Notification Service For more information, refer to [Notification Service API Specification](api-notification-api.html). ### Registering a mobile device instance A mobile device should be registered in Notification Service with respect to the above created mobile app entity. This can be done from the mobile app as per the convenience of app developers. Developer can provide an option to choose the preferred language of receiving notifications on the device of the user. These endpoints are accessed in accordance with guidelines from Mobile App Enablement. For more information about Mobile App Enablement, refer to [Developing Mobile Apps](https://documentation.mindsphere.io/MindSphere/howto/howto-develop-mobile-app.html). When registering a mobile application in Developer Cockpit, select the `mdsp:core:nose.mobileappuser` role in "Authorization Management" in order to access API’s from the mobile device. A mobile installation instance is registered using the following endpoint: ```http POST /mobileApps/{id}/instances ``` Sample Request: ```json { "deviceModel": "iPhone X", "deviceOS": "iOS 10", "language": "de", "pushNotificationToken": "7abcd558f29d0c1f048083e2134ad8ea4b3d87d8ae9c038b43c132706ff445f0" } ``` Sample Response: ```json { "id": "59f92744-7bbc-4540-8b78-ee9976befc1a", "deviceModel": "iPhone X", "deviceOS": "iOS 10", "language": "de", "pushNotificationToken": "7abcd558f29d0c1f048083e2134ad8ea4b3d87d8ae9c038b43c132706ff445f0" } ``` The ID generated `appInstanceIds` will be further used for triggering the notification. The following operations are also possible with a registered mobile device instance. - Show all registered mobile app instances for a given mobile app - Edit a specific mobile app instance registration - Delete a specific mobile app instance registration For more information, refer to [Notification Service API Specification](api-notification-api.html). ### Triggering a notification For triggering the prepared notification to specific users or devices, use the following endpoint: ```http POST/multicastPushNotificationJobs ``` In the following sample request, the `mobileAppId` and `appInstanceIds` created in the previous steps are used. Here, `mobileAppId` is the ID of the respective mobile app and recipients contains the mobile instance or Email address of the logged in user of the app in device, where push notification should be sent. Messages can be sent in multiple languages, having English (en) as the default language. The devices will receive the message in their preferred languages, else message will be received in default language (en). Templates can be used for sending messages, in which case the specific template id can be passed to this API along with specific values for template parameters. If message and template, both are specified, message attribute will be used for sending Push notification. Sample Request: ```json { "mobileAppId": "59f92744-7bbc-4540-8b78-ee9976befc1a", "recipients": { "appInstanceIds": [ "59f92744-7bbc-4540-8b78-ee9976befc1a" ], "userEmailAddresses": [ "joe.smith@siemens.com" ] }, "message": { "title": "{ \"en\": \"Siemens MindSphere October 2019\", \"de\": \"Siemens MindSphere Oktober 2019\"}", "text": "{ \"en\": \"Hi there, Welcome to MindSphere!\", \"de\": \"Hallo, Willkommen bei MindSphere!\"}" } } ``` Sample Response: ```json { "id": "59f92744-7bbc-4540-8b78-ee9976befc1a", "status": "queued", "startTime": "2017-07-21T17:32:28.000Z" } ``` ## E-mail Notifications ### Sending e-mail with attachments Notification Service provides the functionality for sending e-mails to a set of target recipients or a broadcast group along with attachments as optional. When using this endpoint, at least one of `recipients` or `broadcastGroup` is required to be provided. The message to be sent in the e-mail can be provided directly through message attribute or passed through a template. When using template, message attribute may not be specified. When creating an e-mail job a message or template is required to be provided. If both attributes are provided, the message attribute is used for sending e-mail. When an email is sent to recipients listed in `broadcastGroup`, the recipients are listed under 'bcc' field. An e-mail with attachments is sent using the following endpoint: ```http POST/emailNotificationJobs ``` Sample Request without template: ```json { "subject": "[Status] Machine Data Analysis", "message": "Machine data analysis completed. Analysis report attached with this email.", "fromApplication": "MachineMonitor", "priority": "Normal", "recipients": [ "plantengineer@mdspcustomer.com", "plantmanager@mdspcustomer.com" ], "broadcastGroup": { "broadcastType" : "All", "subTenantId": "pnlmfg" }, "sendEncrypted": true } ``` The message to be sent in the e-mail can be provided directly through message attribute or passed through a template. When using template, message attribute may not be specified. When creating an e-mail job, a message or template is required to be provided. If both attributes are provided, the message attribute is used for sending email. Sample Request with template: ```json { "subject": "[Status] Machine Data Analysis", "templateId": "0bff7e7a-cd25-4576-9908-4180ef086174", "templateParams": [{"FactoryName": "MachineFactory"}], "fromApplication": "MachineMonitor", "priority": "Normal", "recipients": [ "plantengineer@mdspcustomer.com", "plantmanager@mdspcustomer.com" ], "broadcastGroup": { "broadcastType" : "All", "subTenantId": "pnlmfg" }, "sendEncrypted": true } ``` Sample Response: ```json { "id": "59f92744-7bbc-4540-8b78-ee9976befc1a", "fromApplication": "MachineMonitor", "status": "Queued", "startTime": "2021-04-21T17:32:28.00" } ``` The ID generated id is referenced further to request status of an e-mail job as well as the per recipient status of a dispatched e-mail. For more information, refer to [Notification Service API Specification](api-notification-api.html). ### Getting status of an e-mail job The current status of an e-mail job is requested using the following endpoint: ```http GET/emailNotificationJobs/{id} ``` Here, the id created in the previous step is used. Sample Response: ```json { "id": "59f92744-7bbc-4540-8b78-ee9976befc1a", "fromApplication": "MachineMonitor", "status": "DispatchedWithFailures", "startTime": "2021-04-21T17:32:28.00", "maliciousAttachments": [ { "filename": "app.pdf", "threat": "worm found" } ] } ``` The following statuses are available: - `Queued` - `InProgress` - `Dispatched` - `DispatchedWithFailures` - `Failed` The system will set the status `dispatchedWithFailures` in two cases: - One or more attachments found infected. In that case, e-mail with message content without attachments is sent to the recipients with a disclaimer. - Delivery to some of the recipients are failed (including bounced e-mails). For more information, refer to [Notification Service API Specification](api-notification-api.html). When email is sent to recipients listed in `broadcastGroup`, the recipients are listed under 'bcc' field. ### Getting delivery information of an e-mail job The per recipient status of a dispatched e-mail is requested using the following endpoint: ```http GET/emailNotificationJobs/{id}/deliveries ``` Here, the `id` created in the previous step is used. Sample Response: ```json { "deliveries": [ { "recipient": "plantengineer@mdspcustomer.com", "status": "Failed" } ], "page": { "size": 0, "totalElements": 0, "totalPages": 0, "number": 0 } } ``` ### Getting list of e-mail notification jobs A list of email notification jobs can be requested using the following endpoint: ```http GET/emailNotificationJobs ``` This list can be further filtered by providing a filter criteria. Sample Response: ```json { "emailNotificationJobs": [ { "id": "59f92744-7bbc-4540-8b78-ee9976befc1a", "fromApplication": "MachineMonitor", "status": "DispatchedWithFailures", "startTime": "2021-04-21T17:32:28.00", "maliciousAttachments": [ { "filename": "app.pdf", "threat": "worm found" } ] } ], "page": { "size": 0, "totalElements": 0, "totalPages": 0, "number": 0 } } ``` The following statuses are available: - `Dispatched` - `Failed` ### Managing e-mail encryption certificates The below endpoint can be used to get a list of email encryption certificates to validate if certificates for all the recipients are available in the system, before sending an encrypted email: ```http GET/emailEncryptionCertificates ``` Sample Response: ```json { "emailEncryptionCertificates": [ { "email": "plantengineer@mdspcustomer.com" } ], "page": { "size": 0, "totalElements": 0, "totalPages": 0, "number": 0 } } ``` Use the following endpoint to get the encryption certificate for a specific recipient: ```http GET/emailEncryptionCertificates/{email} ``` Use the following endpoint to upload a new encryption certificate for a specific recipient: ```http PUT/emailEncryptionCertificates/{email} ``` Use the following endpoint to update the encryption certificate for a specific recipient: ```http PATCH/emailEncryptionCertificates/{email} ``` Use the following endpoint to delete the encryption certificate for a specific recipient: ```http DELETE/emailEncryptionCertificates/{email} ``` ### Getting bounced recipients The below endpoint can be used to get all the recipients to whom the email delivery failed/bounced: ```http GET/bouncedRecipients ``` Sample Response: ```json { "bouncedRecipients": [ { "email": "plantengineer@mdspcustomer.com", "blacklistedDate": "2018-12-11" } ], "page": { "size": 0, "totalElements": 0, "totalPages": 0, "number": 0 } } ``` The recipients to whom the delivery failed/bounced are added to a blacklist. ### ~~Sending e-mail with attachments~~ (Deprecated) Notification Service provides the functionality for sending e-mails to a set of target recipients along with attachments as optional. The only file types supported for e-mail attachments are .pdf, .csv, .json and .zip. An e-mail with attachments is sent using the following endpoint: ```http POST /multicastEmailNotificationJobs ``` Sample Request: Here, `metadata` and `attachment` are `formdata` properties. ```javascript { metadata = { subject: "[Status] Machine Data Analysis", message: "Machine data analysis completed. Analysis report attached with this email.", fromApplication: "MachineMonitor", priority: "Normal", recipients: [ plantengineer@mdspcustomer.com, plantmanager@mdspcustomer.com ] } } attachment = "Multiple Binary Files to be uploaded as attachment" ``` Sample Response: ```json { "id": "59f67823-5bhc-4780-8b23-ee8876mbtg9a" } ``` The ID generated `id` is referenced further to request status of an e-mail job as well as the per recipient status of a dispatched e-mail. For more information, refer to [Notification Service API Specification](api-notification-api.html). ### ~~Getting status of an e-mail job~~ (Deprecated) The current status of an e-mail job is requested using the following endpoint: ```http GET /multicastEmailNotificationJobs/{id} ``` Here, the `id` created in the previous step is used. Sample Response: ```json { "dispatchStatus": [ { "recipient": "plantengineer@mdspcustomer.com", "status": "failed" } ], "page": { "size": 0, "totalElements": 0, "totalPages": 0, "number": 0 } } ``` The following statuses are available: - `queued` - `dispatched` - `dispatchedWithFailures` The system will set the status `dispatchedWithFailures` in two cases: - One or more attachments found infected. In that case, e-mail with message content without attachments is sent to the recipients with a disclaimer. - Delivery to some of the recipients are failed (including bounced e-mails). For more information, refer to [Notification Service API Specification](api-notification-api.html). ### ~~Getting delivery information of an e-mail job~~ (Deprecated) The per recipient status of a dispatched e-mail is requested using the following endpoint: ```http GET/multicastEmailNotificationJobs/{id}/deliveries ``` Here, the `id` created in the previous step is used. Sample Response: ```json { "id": "59f92744-7bbc-4540-8b78-ee9976befc1a", "fromApplication": "MachineMonitor", "status": "queued", "startTime": "2017-07-21T17:32:28.000Z", "maliciousAttachments": [ { "filename": "app.pdf", "threat": "worm found" } ] } ``` ## SMS notifications ### Sending SMS notifications Notification Service provides the functionality for sending SMS notifications to a set of target recipients. Note - Due to regulatory restrictions, sending SMS using the notification service is suspended if at least one of the target recipients is from the India (+91), the USA (+1) or the UK (+44) region. Recipients from these regions shall use other notification options (e-mail or push notification). Templates can be used for sending messages, in which case the specific template id can be passed to this API along with specific values for template parameters. If message and template, both are specified, message attribute will be used for sending SMS notification. An SMS is sent using the following endpoint: ```http POST /multicastSMSNotificationJobs ``` Sample Request: ```json { "message": "Monthly machine data analysis completed.", "recipients": [ "+411xxxxxxx9", "+491xxxxxxx9" ], "fromApplication": "string" } ``` Sample Response: ```json { "id": "87e67823-5cha-5783-8b23-re8876m9ch4a" } ``` The ID generated `id` is referenced further to request status of an SMS job as well as the per recipient status of a dispatched SMS. For more information, refer to [Notification Service API Specification](api-notification-api.html). ### Getting status of an SMS job The current status of an SMS job is requested using the following endpoint: ```http GET /multicastSMSNotificationJobs/{id} ``` Here, the `id` created in the previous step is used. Sample Response: ```json { "id": "59f92744-7bbc-4540-8b78-ee9976befc1a", "fromApplication": "MachineMonitor", "status": "queued", "startTime": "2017-07-21T17:32:28.000Z" } ``` The following statuses are available: - `queued` - `dispatched` - `Not Implemented` For more information, refer to [Notification Service API Specification](api-notification-api.html). ### Getting delivery information of an SMS job The per recipient status of a dispatched SMS is requested using the following endpoint: ```http GET/ /multicastSMSNotificationJobs/{id}/deliveries ``` Here, the `id` created in the previous step is used. Sample Response: ```json { "dispatchStatus": [ { "recipient": "+491xxxxxxx9", "status": "dispatched" }, { "recipient": "+411xxxxxxx9", "status": "failed" } ], "page": { "size": 20, "totalElements": 2, "totalPages": 1, "number": 0 } } ``` ## Templates ### Creating templates Notification Service provides the functionality for using templates to send messages via e-mail, push notification or SMS notification types. For creating a new template definition, use the following endpoint: ```http POST/templates ``` Sample Request: When creating templates E-mail and SMS type of notifications, the place holders defined in the templates need to be specified as `params` in the api `metadata` attribute. Given below is a sample template for reference: ```http Dear PlantEngineer, The regular maintenance of the machines is completed for %FactoryName%. Please plan to start the assembly line. Kind regards, Maintenance Engineer ``` Here, `FactoryName` is the `formdata` property under the metadata attribute. Also, a template file needs to be attached under the `templateFile` attribute. The maximum size of the e-mail template is limited to 8 MB. However, the e-mail message resulting from the E-mail template along with its attachments is limited by the e-mail size limit, that is, 8 MB. In case of SMS, the SMS message resulting from the template is limited to maximum size of SMS notification, that is, 1500 bytes. The supported filetypes for various notifications are listed below: | Notification type | Supported file type(s) | | ----------------- | ---------------------- | | E-mail | .html, .txt | | SMS | .txt | | Push notification | .json | Templates for e-mail and SMS notification types should follow the model as described in `definitions/CreateTemplate`. Similarly, for push notification, follow the model available in `definitions/PushMessageModel`. For more information, refer to [Notification Service API Specification](api-notification-api.html). Sample Response: ```json { "name": "FactoryAlertTemplate", "id": "0bff7e7a-cd25-4576-9908-4180ef086174", "templateFileName": "FactoryNotificationTemplate.html", "params": [ { "FactoryName": "MachineFactory" } ], "type": "Email", "eTag": "1" } ``` The ID generated `id` is referenced further to display, update, and delete a template. ### Getting list of templates A list of templates can be requested using the following endpoint: ```http GET/templates ``` This list can be further filtered by providing a filter criteria. Sample Response: ```json { "Templates": [ { "name": "FactoryAlertTemplate", "id": "0bff7e7a-cd25-4576-9908-4180ef086174", "templateFileName": "FactoryNotificationTemplate.html", "params": [ { "FactoryName": "MachineFactory" } ], "type": "Email", "eTag": "1" } ], "page": { "size": 0, "totalElements": 0, "totalPages": 0, "number": 0 } } ``` ### Managing a template Here, the `id` generated during the template creation is used. Details of a specific template is requested using the following endpoint: ```http GET/templates/{id} ``` Sample Response: ```json { "name": "FactoryAlertTemplate", "id": "0bff7e7a-cd25-4576-9908-4180ef086174", "templateFileName": "FactoryNotificationTemplate.html", "params": [ { "FactoryName": "MachineFactory" } ], "type": "Email", "eTag": "1" } ``` An existing template definition can be updated using the following endpoint: ```http PATCH/templates/{id} ``` Here, the new template definition file provided with the request overrides the previous definition file. When updating a template definition, the maximum size limit for e-mail and SMS type notifications must be within the limit as described in [Creating templates](#creating-templates). An existing template definition can be deleted using the following endpoint: ```http DELETE/templates/{id} ``` It is possible to download an existing template file for review suing the following endpoint: ```http GET/templates/{id}/file ``` # Notification Service – API Overview Region Europe 1 New [Download OpenAPI Specification](/insights-hub/api_specs/notification-fds-v4-4-2-xcelerator.yaml) Old [Download OpenAPI Specification](/insights-hub/api_specs/notification-fds-v3-4-1-xcelerator.yaml) # Notification Service ## Idea Insights Hub provides many functionalities to generate lots of valuable information. Often, these need to be transported quickly to operators or customers so that they can react in time. The Notification Service provides interfaces to communicate and share information among the users of Insights Hub via e-mail, push notification or SMS. Notification Service enables the functionality for using templates to send messages via these notification types. E-mails can be supported with attachments as an optional feature. ## Access - The Notification Service API can only be accessed by third-party applications using a technical authorization token issued by the [Token Management Service](../exchange-tokenmanager/api-tokenmanager-overview.html). Note that the `hosttenant` and the `usertenant` must be set to the tenant the application runs on. - Notification Service specific roles/groups listed in [Notification Service roles and scopes](https://documentation.mindsphere.io/MindSphere/concepts/concept-roles-scopes.html#notification-service) are not explicitly available except `mdsp:core:nose.mobileappuser` role, which is available in authorization management of Developer Cockpit . ## Basics The notification service uses different controllers for the communication and information transmission. Each controller offers various tasks for message processing. The following list describes the content of each individual controller: - **Communication Channel**\ This resource gives information about available communication channels. It gets a list of all active communication types with the channel ID and the channel name such as e-mail and push notifications. - **Address Type**\ This resource lists all address types supported by recipient service such as personal mail, office mail or push notification. - **Recipient**\ With this API controller, you can create a new recipient using the e-mail and phone details and manage the accounts by using an API call. You can execute a search by recipient name and get a recipient based on the recipient ID. Thymeleaf is used as an HTML template engine. Hence, templates are expected to use thymeleaf tags. - **Certificate Store**\ The Certificate Store controller manages the recipient's certification system. You can update, check, retrieve and delete a recipient's certificate. - **Template Param**\ This resource gives information about template parameters. You can use this API to get available template parameters for a requested template set. - **Template Manager**\ The Template Manager allows you to use templates for sending messages such as SMS, e-mail and push notification. You can also merge template parameters in an existing template. Via the API you can get a list of available templates and view the template details and the contents. Each template has a unique ID, which is used to update, delete, obtain details of the template. - **Communication Category**\ The Communication Category controller manages the communication categories. Every category has an ID. You can create and delete the categories. The communication category controller allows you to manage your recipients into different categories. You can also unsubscribe the recipients from a category. For example, a technical support can create a category using recipients and template to define one communication category. - **Encryption Service**\ The encryption service controller encrypts the CcMail, e-mail and plain text. If any of the recipients for a triggered message does not have the respective certificate, the e-mail notification is sent as unencrypted to all the recipients. PGP or S/MIME encrypted e-mails are signed with `noreply@mindsphere.io`. Users can download the public key of the PGP certificate from for installing it in their e-mail client. - **Message Publisher**\ The Message Publisher controller is the basic component of the notification service. It publishes the messages to the queue for further processing and routing to the appropriate channel. - **Communication Service Audit**\ The Communication Service Audit controller saves the message in a database and logs the message information in an audit file. You can use the API to search for the stored audits in the database. The messages are stored with an audit log which would be available up to three months. ## Features The Notification Service exposes its API for realizing the following tasks: - Digital Certificate Management: Upload/update/delete public certificates for the e-mail encryption. Expired certificates cannot be uploaded. After a certificate has expired, e-mails are sent without encryption until a valid certificate has been uploaded again. - Template management: Use pre-defined templates for SMS, push notifications, e-mail notification types. - Configuration: Configure HTML template and recipient or recipient group. Use a unique configuration name for the target audience and reuse the configuration. - Security: Use different e-mail encryption mechanisms, e.g. PGP or S/MIME. Encrypt the messages using different public certificate type of the intended recipients. - Audit logs: Trace the history of the sent notifications via audit log. - SMS Notifications: Send SMS notifications to a set of target recipients. - Broadcast feature for e-mail channel: Send unencrypted e-mails (BCCed) to all users of a tenant including subtenant users or to a specific user group. - Push Notification: Mobile applications (Android or iOS) must be registered in FCM/APNS to receive push notifications. - E-mail attachment: Send e-mails to a set of target recipients along with attachments as optional. ## Limitations & Restrictions - E-mail, SMS and Push notifications shall not be used for time sensitive applications as Insights Hub does not own the total chain of communication. - E-mail addresses for deliveries and templates are treated case insensitively and converted to lower case. - PGP encryption works only for plain text e-mails. E-mails using a pre-defined HTML template would not work correctly as it will display HTML tags after the decryption rather than rendering the HTML tags. - Customers/developers are responsible for the management of their public certificates like expiration of the certificates, replacement of the expired certificates etc. Notification Service will send encrypted e-mails with the expired certificate. However, the actual recipients will not be able to decrypt the e-mail using the expired certificate. - An invalid e-mail address is blacklisted immediately after the first bounce event (refusal). The tenant cannot send further e-mails to blacklisted e-mail addresses.\ If a tenant sends 10 e-mails to invalid addresses within up to 7 days, a warning e-mail is sent to the tenant admins. If the tenant continues to send e-mails to invalid e-mail addresses and the bounce rate of the tenant exceeds 5%, its capability to send e-mails using the Notification Service is disabled. In this case, a tenant admin must contact the support team. - The service may decide to throttle API requests temporarily returning a 429 status code. Note The bounced e-mails are counted as sent from the tenant's account. - Only .pdf, .csv, .json and .zip file types are supported for e-mail attachments. The number of attachments per e-mail is limited to a maximum of 5 files, wherein the total size of all attachments must not exceed 8 MB. This limit is also applicable for an e-mail message (along with its attachments) resulting from a template. - The e-mail attachments are scanned in accordance with the security standards. If one or more attachments are found infected, all the attachments will be removed from the mail. However, the mail with message content will be sent to the recipients with a disclaimer in the footer stating the same. - Depending on the file types and size of the attachments, the scanning can result in a delivery delay of a few minutes because it analyzes each attachment against potential threats. - The maximum number of recipients per e-mail is 50. A recipient is any e-mail address listed in "To", "Cc", or "Bcc". However, a broadcast group is considered as a single recipient and there is no limit for the maximum number of recipients. - E-mails exceeding 250 kB must use a template. - The supported file types for templates are the following: | Notification type | Supported file type(s) | | ----------------- | ---------------------- | | E-mail | .html, .txt | | SMS | .txt | | Push notification | .json | - Maximum of 3 device instances can be added for a user (token email address) of a tenant. - Information about the email job deliveries can only be retrieved up to 180 days from the creation of the email job. After this period, it will be automatically deleted from the system. - The maximum number of recipients per SMS is 50. Note - Due to regulatory restrictions, sending SMS using the notification service is suspended if at least one of the target recipients is from the India (+91), the USA (+1) or the UK (+44) region. Recipients from these regions shall use other notification options (e-mail or push notification). - The SMS messages are scanned for any vulnerabilities in accordance with the security standards before dispatching. If an SMS is found infected, it will not be sent to any of the recipients and the operator receives a 400 status code. - A single SMS message can contain up to 140 bytes of information wherein the character quota depends on the encoding scheme. If the SMS size exceeds 140 bytes, it will be split into multiple messages and sent. When message is split into multiple messages, each partial message will be billed as a single unit. - Maximum size limit for an SMS is 1500 bytes. This limit is also applicable for an SMS message resulting from a template. ## Example Scenario The manager of a wind farm wants to trigger a push notification every time the wind speed exceeds a certain level. The manager uses the API and connects the notification service with the aspect data of the wind turbine. At a wind speed of 8 km/h, the administrator of the wind turbine receives a notification. At a wind speed of 9 km/h, an additional notification is sent to a pre-defined service user list for further action. # Notification Service – Samples for v3.x ## Sending E-Mails with Simple Content Use the following endpoint to send a simple e-mail: ```http POST /api/notification/v3/publisher/messages ``` Sample Request payload: ```json { "body": { "message":"message" }, "recipientsTo":"mail@provider.com;mail@provider-B.com", "from":"name", "subject":"subject" } ``` The `recipientTo` field accepts multiple recipients separated by semicolons. ## Creating E-Mail Templates For sending e-mails using templates, the recipients, the e-mail content and a communication category have to be created first. ### Creating a Recipient A recipient is registered using the following endpoint: ```http POST /api/notification/v3/recipient/ ``` For providing valid input for the `addresstypeid`, refer to the `/recipient/addresstype` of the [Notification Service API Specification](api-notification-api.html). Sample Request: ```json { "recipientname" : "name", "recipientdetail" : [ { "address" : "mail@provider.com", "addresstypeid" : 2 }, { "address" : "mail@provider-B.com", "addresstypeid" : 1 } ] } ``` Sample Response: ```json 806 ``` The response contains the `recipientID` of the created recipient. Store this information as reference for the following API calls. ### Getting recipient details Use the following endpoint get a recipient details based on the `recipientID`: ```http GET /api/notification/v3/recipient/{id} ``` Sample Response: ```json { "recipientid": 124816, "recipientname": "Example recipient", "isactive": "Y", "recipientdetail": [ { "address": "example@siemens.com", "addresstypeid": 2 } ] } ``` Here, `isactive` is the property used to show status (active/inactive) of the recipient. However, currently the value is `Y` (active) by default and it is not possible to change this property. ### Creating a Template for E-Mail Content The template for e-mail content is created using the following endpoint: ```http POST /api/notification/v3/template/ ``` Refer to the API for this endpoint for detailed information on the request structure and creating placeholders. For providing valid input for the `commChannelId`, refer to the `/communicationchannel/` of the [Notification Service API Specification](api-notification-api.html). Sample template: ```html Hello [[${name}]]. Please click the URL for more details : [[${url}]] ``` In this template, the name of the recipient and the included URL are the defined placeholders. Sample response: ```json { "templatesetId": 111, "templatesetName": "TestTemplateForEmail", "templateList": [ { "templateId": 412, "commChannelId": 1, "commChannelName": "Email" } ] } ``` Store the values of `templatesetId` and `templateId` as reference for the following API calls. ### Creating a Communication Category for E-mail A communication category is created using the following endpoint: ```http POST /api/notification/v3/communicationcategories/ ``` In the following sample request, the `recipientID`, `templatesetId` and `templateId` created in the previous steps are reused. Sample Request: ```json { "msgCategoryName": "category name", "subject": "subject", "priority":1, "from":"name", "recipients":[ { "recipientId":807, "position":"TO" } ], "templates":[ { "templateId":412, "templatesetId":111, "commChannelName":"Email" } ] } ``` Sample Response: ```json 642 ``` The response contains the `categoryID` of the generated communication category. Store this information as reference for the following API call. ## Sending E-Mails using a Template For sending e-mails using a template, use the following endpoint: ```http POST /api/notification/v3/publisher/messages ``` The request must provide the `categoryId` and define values for the placeholders of the template for the e-mail content. Default recipients for a communication category are defined during its creation, but the `recipientTo` field accepts additional recipients separated by semicolons. Sample Request: ```json { "body" : { "placeholderkey1" : "value1", "placeholderkey2" : "value2", }, "recipientsTo" : "mail@provider.com", "messageCategoryId" :642 } ``` ## Broadcasting E-Mails Use the following endpoint to broadcast unencrypted e-mails to all users of a tenant including subtenant users: ```http POST /api/notification/v3/broadcastEmails?broadcastType=all ``` The recipients of broadcasted e-mails are concealed using Bcc. Sample request for broadcasting a simple e-mail: ```json { "body": { "message":"Simple broadcast Email content" }, "from":"name", "subject":"subject" "priority": 3 } ``` Sample request for broadcasting an e-mail using a [template](#creating-e-mail-templates): ```json { "body": { "placeholderkey1" : "value1", "placeholderkey2" : "value2", }, "messageCategoryId" :642, "subject":"subject" } ``` Change the `broadcastType` parameter in the request for broadcasting e-mails to other user groups as explained in the [API Specification](api-notification-api.html). ## Sending SMS Notifications using a Template The Notification Service provides the functionality to send SMS notifications to mobile devices. For sending SMS notifications using templates, the recipients, the message content and a communication category have to be created first. ### Creating a recipient with SMS notification details A recipient is registered using the following endpoint: ```http POST /api/notification/v3/recipient/ ``` For providing valid input for the `addresstypeid`, refer to the `/recipient/addresstype` of the [Notification Service API Specification](api-notification-api.html). Note - Due to regulatory restrictions, sending SMS using the notification service is suspended if at least one of the target recipients is from the India (+91), the USA (+1) or the UK (+44) region. Recipients from these regions shall use other notification options (e-mail or push notification). Sample Request: ```json { "recipientname" : "User", "recipientdetail" : [ { "address" : "+411234567890", "addresstypeid" : 4 }, { "address" : "+497777777777", "addresstypeid" : 5 } ] } ``` Sample Response: ```json 100795 ``` ### Creating a Template for SMS Notification Content The template for SMS content is created using the following endpoint: ```http POST /api/notification/v3/template/ ``` Refer to the API for this endpoint for detailed information on the request structure and creating placeholders. For providing valid input for the `commChannelId`, refer to the `/communicationchannel/` of the [Notification Service API Specification](api-notification-api.html). Sample template: ```html Dear [[${name}]], Welcome to SMS service feature of Communication Service Mindsphere.(Hardware Activation) ``` In this template, the name of the recipient is the defined placeholder. The above html is saved as SMS.html and added to request as `templateFiles` parameter. Sample Request: ```json { "templateParam": [{ "paramName": "Name", "defaultValue": "MindSphere User", "placeHolderName": "name", "paramTypeId": 4 }], "templatesetName": "TemplateForSMS", "templateChannelAndFile": [{ "communicationChannel": 2, "fileName": "SMS.html" }] } ``` Sample Response: ```json { "templatesetId": 19489, "templatesetName": "TemplateForSMS", "templateList": [ { "templateId": 22578, "commChannelId": 2, "commChannelName": "SMS" } ] } ``` Store the values of `templatesetId` and `templateId` as reference for the following API calls. ### Creating a Communication Category for SMS A communication category is created using the following endpoint: ```http POST /api/notification/v3/communicationcategories/ ``` In the following sample request, the `recipientId`, `templatesetId` and `templateId` created in the previous steps are re-used. A subject for the SMS notification must be provided in the respective field of the request. Sample Request: ```json { "msgCategoryName" : "Category for SMS", "recipients" : [ { "recipientId" : 100795 } ], "templates" : [ { "templateId" : 22578, "commChannelName" : "SMS", "templatesetId" : 19489 } ] } ``` Sample Response: ```json 29675 ``` The response contains the `categoryId` of the generated communication category. Store this information as reference for the following API call. ### Triggering the SMS Notification For triggering the prepared notification, use the following endpoint: ```http POST /api/notification/v3/publisher/messages ``` The request must provide the `categoryId` and define values for the placeholders of the template for the SMS content. Default recipients for a communication category are defined during its creation. Sample Request: ```json { "body": { "name": "MDSP User" }, "messageCategoryId": 29675 } ``` ## Sending Push Notifications using a Template The Notification Service provides the functionality to send push notifications to mobile devices. For sending push notifications using templates, the recipients, the content and a communication category have to be created first. ### Creating a recipient with push notification details A recipient is registered using the following endpoint: ```http POST /api/notification/v3/recipient/ ``` For providing valid input for the `addresstypeid`, refer to the `/recipient/addresstype` of the [Notification Service API Specification](api-notification-api.html). The following information must be provided in the `address` field of the request: - `appId`: Unique device and application specific ID issued by the GCM connection servers, which allows the client app to receive messages. - `serverApiKey`: Unique application specific key of an application, which is stored on the app server and provides authorized access to Google services - `applicationName`: Name of the application, that sends the notifications Sample Request: ```json { "recipientname" : "PushNotificationTest", "recipientdetail" : [{ "address" : { "appId":"id", "serverApiKey":"apiKey", "applicationName":"app name" }, "addresstypeid" : 7 } ] } ``` Sample Response: ```json 807 ``` The response contains the `recipientID` of the created recipient. Store this information as reference for the following API calls. ### Creating a Template for Push Notification Content The template for push notification content is created using the following endpoint: ```http POST /api/notification/v3/template/ ``` Refer to the API for this endpoint for detailed information on the request structure and creating placeholders. For providing valid input for the `commChannelId`, refer to the `/communicationchannel/` of the [Notification Service API Specification](api-notification-api.html). Sample template: ```html Hello [[${name}]]. Please click the URL for more details : [[${url}]] ``` In this template, the name of the recipient and the included URL are the defined placeholders. Sample response: ```json { "templatesetId": 436, "templatesetName": "TestTemplateForPushNotification12", "templateList": [ { "templateId": 468, "commChannelId": 3, "commChannelName": "Push Notification" } ] } ``` Store the values of `templatesetId` and `templateId` as reference for the following API calls. ### Creating a Communication Category for Push Notification A communication category is created using the following endpoint: ```http POST /api/notification/v3/communicationcategories/ ``` In the following sample request, the `recipientID`, `templatesetId` and `templateId` created in the previous steps are reused. A subject for the push notification must be provided in the respective field of the request. Sample Request: ```json { "msgCategoryName": "category name", "subject": "subject", "priority":1, "from":"name", "recipients":[ { "recipientId":806, "position":"TO" } ] , "templates":[ { "templateId":468, "templatesetId":436, "commChannelName":"Push Notification" } ] } ``` Sample Response: ```json 642 ``` The response contains the `categoryID` of the generated communication category. Store this information as reference for the following API call. ### Triggering the Push Notification For triggering the prepared notification, use the following endpoint: ```http POST /api/notification/v3/publisher/messages ``` The request must provide the `categoryId` and define values for the placeholders of the template for the push content. Default recipients for a communication category are defined during its creation. Sample Request: ```json { "body" : { "placeholderkey1" : "value1", "placeholderkey2" : "value2", }, "messageCategoryId":642 } ``` ## Removing Blocked E-Mail Addresses from the Blacklist Tenant admins can use the following endpoint: ```http DELETE /api/notification/v3/communicationcategories/bouncedEmailIds ``` Sample Request: ```json { "emailList" : ["mail@provider.com"] } ``` Hint Use the following endpoint to list all blacklisted e-mail addresses: ```http GET /api/notification/v3/communicationcategories/bouncedEmailIds ``` Note IoT Value Plan tenants will not be able to access the blacklisted endpoints directly. These tenants can request for the blacklisted emails using Support Request. # Opcenter Intelligence Service – API Overview ## Region Europe 1 [Download OpenAPI Specification](/insights-hub/api_specs/ocin-proxy-app-v1-0-0-eu1-xcelerator.yaml) # Opcenter Intelligence Service - API Overview ## Idea Opcenter Intelligence allows you to access and analyze production-relevant contextualized data from multiple sources. The integration with Insights Hub enables data-driven decision-making based on MES and IoT data. Opcenter Intelligence API Service can be used to access Opcenter Intelligence data to perform analysis and KPI calculation within the supported Insights Hub services. Specifically, these APIs provide: - The list of KPIs defined in Opcenter Intelligence including their configuration data. - KPI calculation results of: - A single KPI, with an applied filter. - A single KPI, with an applied filter and a grouping criterion. - A single KPI, with an applied filter and a target matrix. - Multiple KPIs, with an applied filter. Note This is only applicable for region Europe 1. ## Access For accessing Opcenter Intelligence Service, you need to have the role indicated in [Opcenter Intelligence Service API roles and scopes.](https://documentation.mindsphere.io/MindSphere/concepts/concept-roles-scopes.html#opcenter-intelligence-api) A user can only interact with objects within their tenant and subtenants. ## Prerequisites The following prerequisites are required and common to all APIs: - The Opcenter Intelligence app is enabled for the tenant. - The Opcenter Intelligence app is configured, the Opcenter Intelligence environment is deployed and data ingested. - In Opcenter Intelligence, a smart view is created, deployed, and configured with the necessary measures and attributes. - Multiple KPIs are created. ## Features Opcenter Intelligence APIs provide access to data stored in the Opcenter Intelligence Manufacturing Data Warehouse. For a detailed list of available entities, refer to the Opcenter Intelligence reference manual on the support center. These APIs also support key operations, including: - Retrieving KPI configurations. - Calculating a single KPI with an applied filter. - Calculating a single KPI with an applied filter and a grouping criterion. - Calculating a single KPI with an applied filter and a target matrix. - Calculating multiple KPIs with an applied filter. Info - For more details on APIs, refer to Opcenter Intelligence reference manual available on the support center. - For more details on how to create KPIs in Opcenter Intelligence, refer to Opcenter Intelligence user manual available on the support center. ## Limitations The following are the limitations of Opcenter Intelligence API: - When executing a query, note that the API supports data filtering and provides a mechanism for paginated access. By default, a query returns a maximum of 1000 records. - The API is accessible from UI and server to server. - Data can be accessed by custom applications and standard tools via REST/OData APIs that are compatible with IAM/Cognito authentication. - Query results are cached for 2 minutes. - Key and Value in the query string must be URL-encoded. ## Related links For more information, refer to the following: - [Opcenter Intelligence Basics](https://documentation.mindsphere.io/MindSphere/apps/opcenter-intelligence/introduction-to-ocin-basics.html) - Opcenter Intelligence documentation is available to customers who sign in to [Support Center](https://support.sw.siemens.com/en-US/) with Insights Hub/ Siemens ID credentials. # Rules Management – API Overview ## Region Europe 1 [Download OpenAPI Specification](/insights-hub/api_specs/rulesapiV4-insightshub-xcelerator.yaml) # Rules Management Service ## Idea Rules management allows you to configure, read, delete, and update rules. It is mainly designed for the Insights Hub Monitor Rules wizard, but allows other applications to directly manage rules by calling the respective endpoints. Note This is only applicable for Region Europe 1. ## Access For accessing Rules Management service you need to have the respective roles listed in [Insights Hub Monitor Roles and Scopes](https://documentation.mindsphere.io/MindSphere/apps/insights-hub-monitor/user-rights.html). ## Basics A rule is always defined for multiple assets of the one asset type and contains a list of evaluations. An evaluation is an expression that evaluates to be true or false. If all evaluations evaluate to true for one asset's properties, then all active actions included in the rule are triggered if the rule itself is active. An evaluation includes a property (which reflects an aspect variable), a condition, and comparative values. They can also include additional instructions called "hysteresis" and "debouncing" to define how to handle spikes or other timeseries anomalies. ## Features Rules Management exposes its API for realizing the following tasks: - List the Rules - Create Rules - Update Rules - Delete Rules ## Related links For more information and examples, refer to the following: - [Rules](https://documentation.mindsphere.io/MindSphere/apps/insights-hub-monitor/introduction-rules.html) # Traceability – API Overview ## Region Europe 1 [Download OpenAPI Specification](/insights-hub/api_specs/traceability-app-v1-0-0-eu1-xcelerator.yaml) # Traceability Service - API Overview ## Idea The purpose of the Traceability application is to navigate MES production data to help customers understand the complex relationships between incoming and outgoing materials, operations performed on materials, and the equipment, operators and tools involved in the material processing. This application also explores the genealogy of each material and traces its origin even when the manufacturing process has been executed in different plants. Note This is only applicable for region Europe 1. ## Access To access the Traceability service, you need to have the role indicated in [Opcenter Intelligence Service API roles and scopes.](https://documentation.mindsphere.io/MindSphere/concepts/concept-roles-scopes.html#traceability-api) A user can only interact with objects within their tenant and subtenants. ## Features Traceability APIs support the application’s containment and release functionalities. They enable integration with external systems by providing access to headers and items that are either contained or released. Additionally, two dedicated APIs handle the acknowledgment action, which confirms the reading of containment or release requests. In the “Settings” section of the application, you can set the maximum number of items within a containment request that can be downloaded via the APIs in a single round trip. This value represents the maximum number of items contained in the JSON file when programmatically accessing the APIs used to download an active containment request. Considering typical cloud architecture limitations (connections active for a limited time, typically 30 seconds) and the size of the returned package (usually less than 10 MB), you can define a custom value for the number of elements in a single request. Traceability APIs support pagination when the dataset exceeds the defined limit, allowing sequential calls until the entire dataset is retrieved. ## Limitations The following are the limitations of Traceability APIs: - When executing a query, note that the API supports data filtering and provides a mechanism for paginated access. By default, a query returns a maximum of 10,000 records. You can adjust the limits. Refer to Opcenter Intelligence configuration for the maximum limit, and to the Traceability "Settings" section for the minimum limit. - The APIs are accessible from UI and server to server. - Data can be accessed by custom applications and standard tools via REST/OData API that are compatible with IAM/Cognito authentication. - Query results are cached for 2 minutes. - Key and Value in the query string must be URL-encoded. ## Related links For more information, refer to [Traceability app documentation](https://documentation.mindsphere.io/MindSphere/apps/traceability/introduction.html) # Visual Flow Creator Service – API Overview ## Region Europe 1 [Download OpenAPI Specification](/insights-hub/api_specs/vfc-api-insights-hub-xcelerator.yaml) # Visual Flow Creator Service ## Idea Visual Flow Creator is a software for visual data flow programming in IoT (Internet of Things). Nodes are predefined blocks of functionality. You can interlink and add nodes in the working area and combine them to produce a "flow". Visual Flow Creator API service can be used to create, update, retrieve, delete and other functionalities for the data flows. Note This is only applicable for Region Europe 1. ## Access For accessing Visual Flow Creator service you need to have the respective roles listed in Visual Flow Creator Services. A user can only interact with objects within their tenant and subtenants. ## Basics The Visual Flow Creator API service can be used to create, update, retrieve and delete projects and nodes which belong to projects. It also supports triggering nodes of type `inject` which means that users can directly start flows using the API. ## Features Visual Flow Creator exposes its API for realizing the following tasks: - Projects management - Nodes management - Flows triggering ## Limitations The following are the limitations of Visual Flow Creator: - [Technical Specification](https://documentation.mindsphere.io/MindSphere/apps/visual-flow-creator/appendix.html) ## Example Scenario You can integrate Visual Flow Creator API to your own application in order to access the specific VFC functionality and execute some of the preset flows which are useful for your application. ### Create a Project You can create a project with the provided project name using the `POST` request. For more information, see [Create Project](https://documentation.mindsphere.io/MindSphere/apis/advanced-visual-flow-creator/api-visual-flow-creator-api.html). ### Get the list projects of the specified user You can return a list of projects of the given user using the `GET` request. If no user is specified, an error will be returned. For more information, see [Get Projects List](https://documentation.mindsphere.io/MindSphere/apis/advanced-visual-flow-creator/api-visual-flow-creator-api.html). ## Related links For more information and examples, refer to the following: - [Visual Flow Creator System Manual](https://documentation.mindsphere.io/MindSphere/apps/visual-flow-creator/introduction.html) - [VFC Examples](https://github.com/mindsphere/vfc-examples) # Data Exchange Service - API Overview ## Region Europe 1 [Download OpenAPI Specification](/insights-hub/api_specs/dataexchange-v3-0-2-xcelerator.yaml) # Data Exchange - Usage quota and limits Data Exchange APIs are restricted with following limits. | **API Endpoint** | **Predictive Learning Essentials** | | --------------------------------------------------------- | ---------------------------------- | | core_dataexchange_v3_files_post_60s | 60 | | core_dataexchange_v3_files_post_900s | 300 | | core_dataexchange_v3_files-~\_get_60s | 60 | | core_dataexchange_v3_files-~\_get_900s | 300 | | core_dataexchange_v3_files-~\_put_60s | 60 | | core_dataexchange_v3_files-~\_put_900s | 300 | | core_dataexchange_v3_files-~\_delete_60s | 60 | | core_dataexchange_v3_files-~\_delete_900s | 300 | | core_dataexchange_v3_files-~\_properties_get_60s | 60 | | core_dataexchange_v3_files-~\_properties_get_900s | 300 | | core_dataexchange_v3_files-~\_properties_patch_60s | 60 | | core_dataexchange_v3_files-~\_properties_patch_900s | 300 | | core_dataexchange_v3_directories_post_60s | 60 | | core_dataexchange_v3_directories_post_900s | 300 | | core_dataexchange_v3_directories-~\_properties_get_60s | 60 | | core_dataexchange_v3_directories-~\_properties_get_60s | 300 | | core_dataexchange_v3_directories-~\_properties_patch_60s | 60 | | core_dataexchange_v3_directories-~\_properties_patch_900s | 300 | | core_dataexchange_v3_directories-~\_get_60s | 60 | | core_dataexchange_v3_directories-~\_get_900s | 300 | | core_dataexchange_v3_directories-~\_delete_60s | 60 | | core_dataexchange_v3_directories-~\_delete_900s | 300 | # Data Exchange Service ## Idea The Data Exchange (DE) Service allows customers to upload and download data for any purpose. It is intended for both regular users and for tools that upload data into Insights Hub. ## Access The Data Exchange provides its operations as a REST API. Uploading, downloading, organizing and listing folder structure can be done using API calls. For accessing this service you need to have one of the roles listed in [Data Exchange roles and scopes](https://documentation.mindsphere.io/MindSphere/concepts/concept-roles-scopes.html#data-exchange-service). ## Basics Customers and other services can use DE as a file storage and management provider. Note that when receiving requests through MindGate, MindGate applies its own limits in addition. This includes the timeouts and request sizes. Currently the maximum supported file size is 100 MB. DE does not cache, analyze or scans the contents of the transferred bytes for viruses. It uses the Spring Cloud Resource Manager along with streaming mechanisms that provide multithreaded support for working with S3 storage. This type of backend communication was implemented to ensure maximum performance under minimum resource load. ### Visibility spaces By default, when uploading a file, the newly uploaded file inherits its parent's visibility. It can either be public or private. Public visibility means that the file is visible in tenant space. Private visibility means that the file is visible in user space only, meaning that only the user who has uploaded it can download, delete or update it. #### Public space Public space is accessible by all users within the tenant. Files and directories are made public, if they are uploaded or created in public directories. If the file or directory needs to be placed in root, i.e., they have no parent, the `parentId` attribute must be set to `_PUBLIC_ROOT_ID`. #### Private space Private space is accessible by the respective author only. Files and directories are made private, if they are uploaded or created in private directories. If the file or directory needs to be placed in root, i.e., they have no parent, the `parentId` attribute need to be set to `_PRIVATE_ROOT_ID`. ## Features The Data Exchange Service exposes its API for realizing the following tasks: - Secure upload testbed or production data - Download data from Insights Hub that you have previously uploaded - Download results produced by other Industrial IoT services or apps - Have a performant long term storage for your temporary or permanent data, for both users and services - Organize files in folders - Store confidential files at user level, making them inaccessible for other users of your tenant ## Restrictions - File size limits are internally limited to S3 ones, but when using DataExchange via MindGate, MindGate limits apply - File and directory names are limited to 1024 characters - Number of objects within a parent is limited to 10,000 items - Recursive listing of directory contents is not provided - All environment have 100 GB storage allocated by default irrespective of the offering ## API Rate Limits for P&P Tenant Data Exchange has imposed technical limits for P&P tenant to safeguard the system and to avoid system exploitation on heavy load exceeding system limits.[API rate limits](api-dataexchange-apiratelimits.html) for Data Exchange are applicable as technical rate limits. ## Example Scenario A developer wants to train a model for anomaly detection using the Anomaly Detection Service. The developer uses the Data Exchange Service to upload a training set into Insights Hub. This data can then be used by the Anomaly Detection Service. ## Related Links - [Spring Cloud AWS](https://awspring.io/) # Data Exchange Service – Samples The examples below exemplify how to use the provided endpoints: In order to specify the visibility of a resource, refer to [Data Exchange Service Visibility Spaces](api-dataexchange-overview.html#visibility-spaces). ## Uploading a file in the tenant root space The tenant root space is defined as the root group that contains files and folders accessible to all the users within the tenant. Request: ```http POST /api/dataexchange/v3/files X-XSRF-TOKEN: b73deb0d-feb2-4183-b4c6-a96abe62a44d ``` Form data: ```http metadata :{"name": "custom_file_name.txt", "parentId": "_PUBLIC_ROOT_ID", "type":"CSV"} file : ``` Response: ```json { "resourceId": "327e0baa-f5ba-4d24-af11-8d85de8007e9", "name": "custom_file_name.txt", "parentId": "_PUBLIC_ROOT_ID", "type": "CSV", "sizeInBytes": 8584808, "modifiedDate": "2018-07-12T13:11:29.180", "modifiedBy": "a_user@siemens.com" } ``` ## Uploading a file in the user root space The user root space is defined as the root group that contains files and folders only accessible to a specific user within the tenant. Request: ```http POST /api/dataexchange/v3/files X-XSRF-TOKEN: b73deb0d-feb2-4183-b4c6-a96abe62a44d ``` Form data: ```http metadata :{"name": "custom_file_name.txt", "parentId": "_PRIVATE_ROOT_ID", "type":"CSV"} file : ``` Response: ```json { "resourceId": "327e0baa-f5b1-4df4-af11-8d85de8007e9", "name": "custom_file_name.txt", "parentId": "_PRIVATE_ROOT_ID", "type": "CSV", "sizeInBytes": 8584808, "modifiedDate": "2018-07-12T13:11:29.180", "modifiedBy": "a_user@siemens.com" } ``` ## Creating a directory in the tenant root space The tenant root space is defined as the root group that contains files and folders accessible to all the users within the tenant. Request: ```http POST /api/dataexchange/v3/directories X-XSRF-TOKEN:cade03ca-370c-49fd-9acd-b879368263d6 metadata :{"name": "public_directory", "parentId": "_PUBLIC_ROOT_ID"} ``` Response: ```json { "resourceId": "1821dcbb-13c2-4112-b4c3-0cbb799dce54", "name": "public_directory", "parentId": "_PUBLIC_ROOT_ID", "modifiedDate": "2018-07-12T13:11:29.180", "modifiedBy": "a_user@siemens.com" } ``` ## Creating a directory in the user root space The user root space is defined as the root group that contains files and folders only accessible to a specific user within the tenant. Request: ```http POST /api/dataexchange/v3/directories X-XSRF-TOKEN:cade03ca-370c-49fd-9acd-b879368263d6 metadata :{"name": "private_directory", "parentId": "_PRIVATE_ROOT_ID"} ``` Response: ```json { "resourceId": "1821dcb1-13c2-4112-b4c3-0cbb799dce54", "name": "private_directory", "parentId": "_PRIVATE_ROOT_ID", "modifiedDate": "2018-07-12T13:11:29.180", "modifiedBy": "a_user@siemens.com" } ``` ## Deleting a file Request: ```http DELETE /api/dataexchange/v3/files/327e0baa-f5b1-4df4-af11-8d85de8007e9 HTTP/1.1 X-XSRF-TOKEN: b73deb0d-feb2-4183-b4c6-a96abe62a44d ``` ## Deleting a directory Request: ```http DELETE /api/dataexchange/v3/directories/1821dcbb-13c2-4112-b4c3-0cbb799dce54 HTTP/1.1 X-XSRF-TOKEN: b73deb0d-feb2-4183-b4c6-a96abe62a44d ``` ## Listing a directory contents Request: ```http GET /api/dataexchange/v3/directories/_PRIVATE_ROOT_ID ``` ```http GET /api/dataexchange/v3/directories/_PUBLIC_ROOT_ID ``` ```http GET /api/dataexchange/v3/directories/1821dcbb-13c2-4112-b4c3-0cbb799dce54 ``` Response: ```json { "files": [ { "id": "f6214934-6f2f-416f-87e1-1fee5c31d91b", "parentId": "1821dcbb-13c2-4112-b4c3-0cbb799dce54", "name": "some_file.txt", "type": "TXT", "modifiedBy": "a_user@siemens.com", "sizeInBytes": 8584808, "lastModified": "2018-07-12T13:07:26.966" }, { "id": "c8a849e2-801c-47f0-b648-4efb41101b78", "parentId": "1821dcbb-13c2-4112-b4c3-0cbb799dce54", "name": "another_file.csv", "type": "CSV", "modifiedBy": "some.name@siemens.com", "sizeInBytes": 563433, "lastModified": "2018-07-09T13:53:36.213" } ], "directories": [ { "id": "17ee9f30-501c-4efa-bdcd-8bbf52d87941", "parentId": "1821dcbb-13c2-4112-b4c3-0cbb799dce54", "name": "dir1", "modifiedBy": "some.name@siemens.com" }, { "id": "1891dcbb-13c2-4111-b4c3-0cbb799dce54", "parentId": "1821dcbb-13c2-4112-b4c3-0cbb799dce54", "name": "dir2", "modifiedBy": "a_user@siemens.com" } ] } ``` Note The `parentId` field accepts the values of `_PUBLIC_ROOT_ID` or `_PRIVATE_ROOT_ID` or the resource ID for which the listing is performed. ## Retrieving a directory's properties Request: ```http GET /api/dataexchange/v3/directories/327e0baa-f5b1-4df4-af11-8d85de8007e9 ``` Response: ```json { "resourceId": "327e0baa-f5b1-4df4-af11-8d85de8007e9", "name": "public_dir.txt", "parentId": "_PUBLIC_ROOT_ID", "modifiedDate": "2018-07-12T13:11:29.180", "modifiedBy": "a_user@siemens.com" } ``` ## Retrieving a file's properties Request: ```http GET /api/dataexchange/v3/files/327e0baa-f5b1-4df4-af11-8d85de8007e9/properties ``` Response: ```json { "resourceId": "327e0baa-f5b1-4df4-af11-8d85de8007e9", "name": "custom_file_name.txt", "parentId": "327e0baa-f5b1-4df4-af11-8d85de8007e9", "type": "CSV", "sizeInBytes": 8584808, "modifiedDate": "2018-07-12T13:11:29.180", "modifiedBy": "a_user@siemens.com" } ``` ## Downloading a file Request: ```http GET /api/dataexchange/v3/files/327e0baa-f5b1-4df4-af11-8d85de8007e9 Content-Type: application/json Accept: application/octet-stream ``` Response: ```json ``` ## Updating/Replacing a file's content Request: ```http GET /api/dataexchange/v3/files/327e0baa-f5b1-4df4-af11-8d85de8007e9 X-XSRF-TOKEN: b73deb0d-feb2-4183-b4c6-a96abe62a44d Accept: application/octet-stream ``` Form data: ```http file: ``` Response: ```json { "resourceId": "327e0baa-f5b1-4df4-af11-8d85de8007e9", "name": "custom_file_name.txt", "parentId": "327e0baa-f5b1-4df4-af11-8d85de8007e9", "type": "CSV", "sizeInBytes": 8584808, "modifiedDate": "2018-07-12T13:11:29.180", "modifiedBy": "a_user@siemens.com" } ``` ## Renaming/Moving/Updating type or visibility for a file If a file's visibility is public, the file is visible for all users within the tenant. A file can be made public by the following actions: - Placing it into the \_PUBLIC_ROOT_ID - Placing it in a directory that is public Analogously, use private folders for making a file private. Request: ```text PATCH /api/dataexchange/v3/files/327e0baa-f5b1-4df4-af11-8d85de8007e9 X-XSRF-TOKEN: b73deb0d-feb2-4183-b4c6-a96abe62a44d Accept: application/octet-stream ``` Body: ```json {"name": "renamed_filename.txt", "parentId":"_PUBLIC_ROOT_ID", "type":"Parquet"} ``` Response: ```json { "resourceId": "327e0baa-f5b1-4df4-af11-8d85de8007e9", "name": "custom_file_name.txt", "parentId": "_PUBLIC_ROOT_ID", "type": "Parquet", "sizeInBytes": 8584808, "modifiedDate": "2018-07-12T13:11:29.180", "modifiedBy": "a_user@siemens.com" } ``` ## Renaming/Moving/Updating visibility for a directory If a directory's visibility is public, the file is visible for all users within the tenant. A directory can be made public by the following actions: - Placing it into the \_PUBLIC_ROOT_ID - Placing it in a directory that is public Analogously, use private folders for making a directory private. Request: ```http PATCH /api/dataexchange/v3/directories/327e0baa-f5b1-4df4-af11-8d85de8007e9 X-XSRF-TOKEN: b73deb0d-feb2-4183-b4c6-a96abe62a44d Accept: application/octet-stream ``` Body: ```json {"name": "renamed_directory", "parentId":"_PUBLIC_ROOT_ID"} ``` Response: ```json { "resourceId": "327e0baa-f5b1-4df4-af11-8d85de8007e9", "name": "renamed_directory", "parentId": "_PUBLIC_ROOT_ID", "modifiedDate": "2018-07-12T13:11:29.180", "modifiedBy": "a_user@siemens.com" } ``` # Job Manager Service - API Overview ## Region Europe 1 [Download OpenAPI Specification](/insights-hub/api_specs/jobmanager-v3-xcelerator.yaml) # Job Manager - Usage quota and limits Job Manager APIs are restricted with following limits. | **API Endpoint** | **Predictive Learning Essentials** | | ----------------------------------------------- | ---------------------------------- | | core_jobmanager_v3_jobs_get_60s | 60 | | core_jobmanager_v3_jobs_get_900s | 300 | | core_jobmanager_v3_jobs_post_60s | 10 | | core_jobmanager_v3_jobs_post_900s | 30 | | core_jobmanager_v3_jobs-~\_get_60s | 60 | | core_jobmanager_v3_jobs-~\_get_900s | 300 | | core_jobmanager_v3_jobs-~\_stop_post_60s | 60 | | core_jobmanager_v3_jobs-~\_stop_post_900s | 300 | | core_jobmanager_v3_schedules_get_60s | 60 | | core_jobmanager_v3_schedules_get_900s | 300 | | core_jobmanager_v3_schedules_post_60s | 60 | | core_jobmanager_v3_schedules_post_900s | 300 | | core_jobmanager_v3_schedules-~\_get_60s | 60 | | core_jobmanager_v3_schedules-~\_get_900s | 300 | | core_jobmanager_v3_schedules-~\_delete_60s | 60 | | core_jobmanager_v3_schedules-~\_delete_900s | 300 | | core_jobmanager_v3_schedules-~\_start_post_60s | 60 | | core_jobmanager_v3_schedules-~\_start_post_900s | 300 | | core_jobmanager_v3_schedules-~\_stop_post_60s | 60 | | core_jobmanager_v3_schedules-~\_stop_post_900s | 300 | # Job Manager Service ## Idea The Job Manager Service allows customers to execute analytical and data science models, which consist of arbitrary paragraphs of code. Currently, the Job Manager Service only supports Apache Zeppelin virtual notebooks. Info The Job Manager Service is currently only available in region Europe 1. ## Access For accessing this service you need to have the respective roles listed in [Job Manager roles and scopes](https://documentation.mindsphere.io/MindSphere/concepts/concept-roles-scopes.html#job-manager-service). ## Basics The Job Manager Service follows an execution pattern, which consists of the following steps: 1. Validate input parameters, their existence and security availability. 1. Start the execution environment. 1. Prepare input data. 1. Execute Zeppelin virtual notebook. 1. Store results in user defined output directory. ### Dependencies #### Model Management Service The Job Manager Service uses the API of the [Model Management Service](../analytics-modelmanagement/api-modelmanagement-overview.html) to access the analytical models to be executed. The respective models must be accessible by the tenant. #### Data Exchange Service The Job Manager Service uses the API of the [Data Exchange Service](../analytics-dataexchange/api-dataexchange-overview.html) to read input data and export output data. The respective locations must be accessible by the tenant. #### Predictive Learning Services The Job Manager Services runs the specified jobs in a virtual environment. This environment must be defined using the [Predictive Learning](https://documentation.mindsphere.io/MindSphere/apps/predictive-learning-essentials/welcome-predictive-learning-help.html) services. ### Apache Zeppelin [Apache Zeppelin](https://zeppelin.apache.org/) is a virtual notebook environment. Apache Zeppelin notebooks support various interpreters, for many languages and frameworks, including Scala, Python, Java etc. They are useful in at least two general scenarios: - training a model to obtain an inference model - performing inference or prediction tasks ## Features Training a model usually requires high computation resources like memory, storage, bandwidth and CPU. The Job Manager Service exposes its API for realizing the following tasks: - Validate provided input before proceeding with expensive operations - Perform necessary cleanups regardless of the success or failure of the execution - Retry expensive operations automatically in case of failure - Record important outputs for the user to backtrace errors - Minimize the usage of expensive resources ## Limitations - All input files have to pass the [Gateway](https://documentation.mindsphere.io/MindSphere/concepts/concept-gateway-url-schemas.html), which has its own limitations. - Each execution is started in a separate execution environment. - Setting up the execution environment can take up to 30 minutes. Keep this in mind for time-sensitive prediction or inference tasks. - The preparation time for input data and results is linearly dependent on the file sizes. ## API Rate Limits for P&P Tenant Job Manager has imposed technical limits for P&P tenant to safeguard the system and to avoid system exploitation on heavy load exceeding system limits.[API rate limits](api-jobmanager-apiratelimits.html) for Job Manager are applicable as technical rate limits. ## Example Scenario A developer wants to train an Apache Zeppelin notebook model for anomaly detection. The developer uses the Job Manager Service to (re)train this model. ## Related Links - [Anomaly Detection](https://en.wikipedia.org/wiki/Anomaly_detection) - [Apache Zeppelin](https://zeppelin.apache.org/) - [Apache Zeppelin Interpreters](https://zeppelin.apache.org/docs/0.8.0/usage/interpreter/overview.html) # Job Manager Service– Samples ## Running a Model ### Input Parameters - `modelId`: Defines the ID of the model, which must be accessible by the tenant using [Model Management Service](../analytics-modelmanagement/api-modelmanagement-overview.html). Supported Models Currently, only Apache Zeppelin notebooks are supported. - `configurationId`: Defines the [Environment Configuration](https://documentation.mindsphere.io/MindSphere/apps/predictive-learning-essentials/environments.html) created using the Predictive Learning services. The Job Manager instantiates the environment and runs the model on it. - `inputFolderId`: Defines the folder holding the input data, which must be accessible by the tenant using [Data Exchange Service](../analytics-dataexchange/api-dataexchange-overview.html). - `outputFolderId`: Defines the output folder, which must already be created and be accessible by the tenant using [Data Exchange Service](../analytics-dataexchange/api-dataexchange-overview.html). - `maximumExecutionTimeInSeconds`: Defines the maximum allowed execution time of the job. Only the actual execution time of the execution engine (Apache Zeppelin) is taken into account. ### Starting the Job Request: ```http POST /api/jobmanager/v3/jobs X-XSRF-TOKEN: `` ``` The input parameters are defined in the body: ```json { "modelId": "", "configurationId": "", "inputFolderId": "", "outputFolderId": "", "maximumExecutionTimeInSeconds":"7200" } ``` Sample response: ```json { "id": "", "modelId": "", "environmentId": "", "message": "", "status": "SUBMITTED", "creationDate": "2018-10-01T12:00:00.001Z", "createdBy": "", "inputFolderId": "", "outputFolderId": "", "configurationId": "", "maximumExecutionTimeInSeconds":"7200" } ``` The `` is required for monitoring the job using the Job Manager. ## Monitoring Job Execution The current status of a job is requested using: ```http GET /api/jobmanager/v3/jobs/ ``` Sample response: ```json { "id": "", "modelId": "", "environmentId": "", "message": "Started notebook execution.", "status": "RUNNING", "creationDate": "2018-10-01T02:00:00.001Z", "createdBy": "", "inputFolderId": "", "outputFolderId": "", "configurationId": "", "maximumExecutionTimeInSeconds":"7200" } ``` The following statuses are available: ```json SUBMITTED STARTING RUNNING STOPPING FAILED SUCCEDED STOPPED ``` The status is `FAILED`, if the job execution cannot be completed or an error occurred during the execution. This is also true, if only a single paragraph within a Zeppelin notebook fails and all other steps succeed. The Job Manager attempts to continue a job execution, if the execution workflow allows skipping individual steps. ## Retrieving the List of Jobs A list of all available jobs is retrieved using the following request: ```http GET /api/jobmanager/v3/jobs ``` By default, the Job Manager divides the result into pages of 100 entries and returns the first page. Using the query parameter ``, the page number to be returned can be changed. The query parameter `` changes the number entries per page. Response: ```json { "content": [ { "jobId": "", "modelId": "", "environmentId": "", "message": "message", "status": "SUCCEEDED", "creationDate": "2018-10-01T12:00:00.001Z", "createdBy": "TenantId", "inputFolderId": "", "outputFolderId": "", "configurationId": "", "maximumExecutionTimeInSeconds":"7200" }, { "jobId": "", "modelId": "", "environmentId": "", "message": "Unable to import model into Zeppelin[I/O error on GET request for \"https://gateway.{region}.{MindSphere-domain}/api/modelmanagement/v3/models//versions/last\": Server returned HTTP response code: 504 for URL: https://gateway.{region}.{MindSphere-domain}/api/modelmanagement/v3/models//versions/last; nested exception is java.io.IOException: Server returned HTTP response code: 504 for URL: https://gateway.{region}.{MindSphere-domain}/api/modelmanagement/v3/models/Model_id/versions/last] Error while stopping environment", "status": "STOPPING", "creationDate": "2018-10-01T12:00:00.001Z", "createdBy": "TenantId", "inputFolderId": "", "outputFolderId": "", "configurationId": "", "maximumExecutionTimeInSeconds":"7200" }, { "jobId": "", "modelId": "", "environmentId": "", "message": "Failed to start environment [An environment has already been started for the configuration]", "status": "FAILED", "creationDate": "2018-10-02T12:00:00.001Z", "createdBy": "TenantId", "inputFolderId": "", "outputFolderId": "", "configurationId": "", "maximumExecutionTimeInSeconds":"7200" } ], "totalPages": 1, "totalElements": 3, "last": true, "size": 20, "number": 0, "numberOfElements": 3, "first": true, "sort": null } ``` The second and third entry illustrate, that the chain of messages is returned to the user, if a fatal error is encountered and the execution logic cannot recover. The Job Manager attempts to stop the used instances, while preserving any outputs that might have been produced. # Model Management Service - API Overview ## Region Europe 1 [Download OpenAPI Specification](/insights-hub/api_specs/modelmanagement-v3-xcelerator.yaml) # Model Management - Usage quota and limits Model Management APIs are restricted with following limits. | **API Endpoint** | **Predictive Learning Essentials** | | ----------------------------------------------------------- | ---------------------------------- | | core_modelmanagement_v3_models_get_60s | 60 | | core_modelmanagement_v3_models_get_900s | 300 | | core_modelmanagement_v3_models_post_60s | 60 | | core_modelmanagement_v3_models_post_900s | 300 | | core_modelmanagement_v3_models-~\_get_60s | 60 | | core_modelmanagement_v3_models-~\_get_900s | 300 | | core_modelmanagement_v3_models-~\_patch_60s | 60 | | core_modelmanagement_v3_models-~\_patch_900s | 300 | | core_modelmanagement_v3_models-~\_delete_60s | 60 | | core_modelmanagement_v3_models-~\_delete_900s | 300 | | core_modelmanagement_v3_models-~\_versions_last_get_60s | 60 | | core_modelmanagement_v3_models-~\_versions_last_get_900s | 300 | | core_modelmanagement_v3_models-~\_versions_last_patch_60s | 60 | | core_modelmanagement_v3_models-~\_versions_last_patch_900s | 300 | | core_modelmanagement_v3_models-~\_versions_last_delete_60s | 60 | | core_modelmanagement_v3_models-~\_versions_last_delete_900s | 300 | | core_modelmanagement_v3_models-~\_versions_post_60s | 60 | | core_modelmanagement_v3_models-~\_versions_post_900s | 300 | | core_modelmanagement_v3_models-~\_versions_get_60s | 60 | | core_modelmanagement_v3_models-~\_versions_get_900s | 300 | | core_modelmanagement_v3_models-\_versions-\_get_60s | 60 | | core_modelmanagement_v3_models-\_versions-\_get_900s | 300 | | core_modelmanagement_v3_models-\_versions-\_delete_60s | 60 | | core_modelmanagement_v3_models-\_versions-\_delete_900s | 300 | # Model Management Service ## Idea The Model Management for Analytical solutions helps customers to store single file models, algorithms, scripts, docker images and training or validation data, used for machine learning or AI tasks. ## Access The Model Management Service is exposed as a REST API. Storing, retrieving, updating of models and their versions, along with the associated metadata can be done by simple API calls. For accessing this service you need to have the respective roles listed in [Model Management Service roles and scopes](https://documentation.mindsphere.io/MindSphere/concepts/concept-roles-scopes.html#model-management-service). ## Basics The Model Management Service stores and serves models for either active users or applications, which require storing of (large) binaries. It supports both versioning and metadata information. Model Management supports structured information associated with models, such as: - **Model Metadata:** Provides general model information - **Version Metadata:** Provides traceability of model versions - **Version Payload:** Provides traceability of the actual binary content of a model, which is always associated with version information ### Model Metadata The model metadata stores general model information like name, author, creation date and its type. ### Version Metadata The version metadata stores detailed information regarding the stored version. Those are the version number, the type (like Zeppelin, Jupyter, Protobuff, Docker etc.), in/out parameters, freeform parameters, build and/or run dependencies (libraries and associated version), and dependencies on other models. A dependency on another model is for example given, if the other model produces a payload, which is required as an input. This dependency is defined using the `producedBy` field. ### Version payload The version payload stores the actual model content in a file, which can be of any type, including .json, .pmml, .py, .ipynb, or .pb. ## Features The Model Management Service exposes its API for realizing the following tasks: - Storing analytical model binaries and versioning info - Managing versions of a model - Downloading a model for examination or execution - Defining dependencies needed to execute a model - Defining parameters required to execute a model ## Limitations - Currently, the Model Management Service can only store one version payload (file) for a specific version of a model. - Model needs to be kept in sync with asset modelling. The user needs to update the model and retain it, if there is any change in asset modelling which is used in the algorithm/model. If this is not done, then jobs using such models will start failing as they would be looking for a variable/aspect/asset by name which does not exists in the system. - All tenants have 100 GB storage allocated by default irrespective of the offering ## API Rate Limits for P&P Tenant Model Management has imposed technical limits for P&P tenant to safeguard the system and to avoid system exploitation on heavy load exceeding system limits.[API rate limits](api-modelmanagement-apiratelimits.html) for Model Management are applicable as technical rate limits. ## Example Scenario A client has their own analytical models for training or forecasting. They use the Model Management Service store these models and retrieve them for training. After the training is finished, the model can output weights or another type of trained model binaries. Additional models or simple inference services can load the weights files to perform predictions. ## Related Links - [Saving Weights to Files](https://machinelearningmastery.com/save-load-keras-deep-learning-models/) - [Model Types for Machine Learning](https://docs.aws.amazon.com/machine-learning/latest/dg/types-of-ml-models.html) - [Start using Docker images](https://docs.docker.com/get-started/) # Model Management Service– Samples The samples below exemplify how to use the provided endpoints. Note For avoiding confusions with JSON syntax, placeholders are indicated using angle brackets instead of curly brackets in the following samples. Note All requests must contain authorization headers in one of the two forms: ```http X-XSRF-TOKEN: `` ``` or ```http Authorization: Bearer `` ``` ## Uploading a Model with Version Metadata and a Binary File Request: ```http POST /api/modelmanagement/v3/models ``` Provide the following content for the `metadata` key of the `form-data`: ```json { "name": "Model 1", "description": "Model 1", "type": "Protobuf file", "lastVersion": { "number": 1.0, "expirationDate": "2019-10-01T12:00:00.001", "dependencies": [{ "name": "sklearn-theano", "type": "Python", "version": "2.7" }], "io": { "consumes": "CSV", "input": [{ "name": "variablename1", "type": "integer", "description": "description for variablename1", "value": 5 }], "output": [{ "name": "outputname1", "type": "integer", "description": "description for outputname1", "value": null }], "optionalParameters": { "freeFormParams": "for the author to use", "param1": "value1" } }, "producedBy": "[, ]", "kpi": [{ "name": "error rate", "value": 0.9 }] } } ``` The `file` key of the `form-data` must contain the payload of the model's binary file. Response: ```json { "id": "", "name": "Model 1", "description": "Model 1", "type": "PDF File", "author": "creator@siemens.com", "lastVersion": { "id": "", "number": 1.0, "expirationDate": "2019-10-01T12:00:00.001", "author": "user@siemens.com", "creationDate": "2018-10-01T12:00:00.001", "dependencies": [{ "name": "sklearn-theano", "type": "Python", "version": "2.7" }], "io": { "consumes": "CSV", "input": [{ "name": "variablename1", "type": "integer", "description": "description for variablename1", "value": 5 }], "output": [{ "name": "outputname1", "type": "integer", "description": "description for outputname1", "value": null }], "optionalParameters": { "freeFormParams": "for the author to use", "param1":"value1" } }, "producedBy": "[, ]", "kpi": [{ "name": "error rate", "value": 0.9 }] } } ``` ## Uploading a New Version of a Model Request: ```http POST /api/modelmanagement/v3/models//versions ``` Provide the following content for the `metadata` key of the `form-data`: ```json { "number": 1.0, "expirationDate": "2019-10-01T12:00:00.001", "dependencies": [{ "name": "sklearn-theano", "type": "Python", "version": "1.7" }], "io": { "consumes": "CSV", "input": [{ "name": "variablename1", "type": "integer", "description": "description for variablename1", "value": 5 }], "output": [{ "name": "outputname1", "type": "integer", "description": "description for outputname1", "value": null }], "optionalParameters": { "freeFormParams": "for the author to use", "param1":"value1" } }, "producedBy": "[, ]", "kpi": [{ "name": "error rate", "value": 0.9 }] } ``` The `file` key of the `form-data` must contain the payload of the model's binary file. Response: ```json { "id": "", "number": 1.0, "expirationDate": "2019-10-01T12:00:00.001", "author": "user@siemens.com", "creationDate": "2018-10-01T12:00:00.001", "dependencies": [{ "name": "sklearn-theano", "type": "Python", "version": "1.7" }], "io": { "consumes": "CSV", "input": [{ "name": "variablename1", "type": "integer", "description": "description for variablename1", "value": 5 }], "output": [{ "name": "outputname1", "type": "integer", "description": "description for outputname1", "value": null }], "optionalParameters": { "freeFormParams": "for the author to use", "param1":"value1" } }, "producedBy": "[, ]", "kpi": [{ "name": "error rate", "value": 0.9 }] } ``` ## Updating a Model The `PATCH` request is used to selectively update specific attributes of a model's metadata and its version. When updating array fields, the entire array must be provided as an input, because the service completely replaces the respective arrays.\ For example, the entire list of items must be provided, when updating the `dependencies` field. The Model Management service takes the items provided in the request and replaces the stored attribute entirely. This also applies to the `io`, `produced_by` and `kpi` fields. Except for `author`, `creationDate`, and `id`, any field of the model can be updated, including its version (attribute `lastVersion`). Note that, the `number` attribute is automatically incremented, if not provided with a higher version than what is stored. Request: ```http PATCH /api/modelmanagement/v3/models/ ``` Provide the following content for the `metadata` key of the `form-data`: ```json { "name":"new model name", "lastVersion": { "number": 1.4, "io": { "consumes": "CSV", "input": [{ "name": "variablename1", "type": "integer", "description": "description for variablename1", "value": 5 }, { "name": "newVarName", "type": "string", "description": "description for newVarName, e.g. path", "value": "/usr/lib/zeppelin/some.c" } ] }, } } ``` The `file` key of the `form-data` must contain the payload of the model's binary file. Response: ```json { "id": "", "name":"new model name", "description": "old model description", "author": "user@siemens.com", "lastVersion": { "id": "", "number": 1.0, "expirationDate": "2019-10-01T12:00:00.001", "author": "user@siemens.com", "creationDate": "2018-10-01T12:00:00.001", "dependencies": [{ "name": "sklearn-theano", "type": "Python", "version": "1.7" }], "io": { "consumes": "CSV", "input": [{ "name": "variablename1", "type": "integer", "description": "description for variablename1", "value": 5 }, { "name": "newVarName", "type": "string", "description": "description for newVarName, e.g. path", "value": "/usr/lib/zeppelin/some.c" } ], "output": [{ "name": "outputname1", "type": "integer", "description": "description for outputname1", "value": null }], "optionalParameters": { "freeFormParams": "for the author to use", "param1":"value1" } }, "producedBy": "[, ]", "kpi": [{ "name": "error rate", "value": 0.9 }] } } ``` ## Updating the Last Version of a Model Request: ```http PATCH /api/modelmanagement/v3/models//versions/last ``` Any field of the version can be updated, except for `author`, `creationDate` and `id`. Provide the following content for the `metadata` key of the `form-data`: ```json { "number": 1.2, "expirationDate": "2019-10-01T12:00:00.001", "dependencies": [{ "name": "sklearn-theano", "type": "Python", "version": "1.7" }, { "name": "numpy", "type": "Python", "version": "1.15" }], "io": { "consumes": "CSV", "input": [{ "name": "variablename1", "type": "integer", "description": "description for variablename1", "value": 5 }], "output": [{ "name": "outputname1", "type": "integer", "description": "description for outputname1", "value": null }], "optionalParameters": { "freeFormParams": "for the author to use", "param1":"value1" } }, "producedBy": "[, ]", "kpi": [{ "name": "error rate", "value": 0.9 }] } ``` The `file` key of the `form-data` must contain the payload of the model's binary file. Response: ```json { "id": "", "number": 1.2, "expirationDate": "2019-10-01T12:00:00.001", "author": "user@siemens.com", "creationDate": "2018-10-01T12:00:00.001", "dependencies": [{ "name": "sklearn-theano", "type": "Python", "version": "1.7" }, { "name": "numpy", "type": "Python", "version": "1.15" }], "io": { "consumes": "CSV", "input": [{ "name": "variablename1", "type": "integer", "description": "description for variablename1", "value": 5 }], "output": [{ "name": "outputname1", "type": "integer", "description": "description for outputname1", "value": null }], "optionalParameters": { "freeFormParams": "for the author to use", "param1":"value1" } }, "producedBy": "[, ]", "kpi": [{ "name": "error rate", "value": 0.9 }] } ``` ## Listing Available Models This lists all available models, along with their last version only. Request: ```http GET /api/modelmanagement/v3/models Accept: `application/json` ``` Response: ```json {[ { "id": "", "name": "some model 1", "description": "Some Model 1", "type": "PB file", "author": "user@siemens.com", "lastVersion": { "id": "", "number": 1.0, "expirationDate": "2019-10-01T12:00:00.001", "author": "user@siemens.com", "creationDate": "2018-10-01T12:00:00.001", "dependencies": [{ "name": "sklearn-theano", "type": "Python", "version": "1.7" }], "io": { "consumes": "CSV", "input": [{ "name": "variablename1", "type": "integer", "description": "description for variablename1", "value": 5 }], "output": [{ "name": "outputname1", "type": "integer", "description": "description for outputname1", "value": null }], "optionalParameters": { "freeFormParams": "for the author to use", "param1":"value1" } }, "producedBy": "[, ]", "kpi": [{ "name": "error rate", "value": 0.9 }] } }, { "id": "", "name": "some model 2", "description": "Some Model 2", "type": "JSON file", "author": "user@siemens.com", "lastVersion": { "id": "", "number": 1.0, "expirationDate": "2019-11-01T12:00:00.001", "author": "user@siemens.com", "creationDate": "2018-10-01T12:00:00.001", "dependencies": [{ "name": "sklearn-theano", "type": "Python", "version": "1.7" }], "io": { "consumes": "CSV", "input": [{ "name": "variablename1", "type": "integer", "description": "description for variablename1", "value": 5 }], "output": [{ "name": "outputname1", "type": "integer", "description": "description for outputname1", "value": null }], "optionalParameters": { "freeFormParams": "for the author to use", "param1":"value1" } }, "producedBy": "[, ]", "kpi": [{ "name": "error rate", "value": 0.9 }] } }], "page": { "number": 0, "size": 14, "totalPages": 1, "totalElements": 14 } } ``` ## Obtaining a Model's Metadata Request: ```http GET /api/modelmanagemen/v3/models/` ``` Response: ```json { "id": "", "name": "some_model", "description": "Model 1", "type": "PDF File", "author": "creator@siemens.com", "lastVersion": { "id": "", "number": 1.0, "expirationDate": "2019-10-01T12:00:00.001", "author": "user@siemens.com", "creationDate": "2018-10-01T12:00:00.001", "dependencies": [{ "name": "sklearn-theano", "type": "Python", "version": "1.7" }], "io": { "consumes": "CSV", "input": [{ "name": "variablename1", "type": "integer", "description": "description for variablename1", "value": 5 }], "output": [{ "name": "outputname1", "type": "integer", "description": "description for outputname1", "value": null }], "optionalParameters": { "freeFormParams": "for the author to use", "param1":"value1" } }, "producedBy": [, ], "kpi": [{ "name": "error rate", "value": 0.9 }] } } ``` ## Downloading a Model's Metadata or Last Version's Payload This request can be used to download either the metadata or the payload of the last version based on the `Content-Type` header. If the `Content-Type` is specified as `application/octet-stream`, the service returns the last version's payload. In this sample the Content-Type is specified as `application/json`, for which the last version's metadata is returned: ```http GET /api/modelmanagement/v3//versions/ HTTP/1.1 Content-Type: `application/json` ``` Response: ```json { "id": "", "number": 1.0, "expirationDate": "2019-10-01T12:00:00.001", "author": "user@siemens.com", "creationDate": "2018-10-01T12:00:00.001", "dependencies": [{ "name": "sklearn-theano", "type": "Python", "version": "1.7" }], "io": { "consumes": "CSV", "input": [{ "name": "variablename1", "type": "integer", "description": "description for variablename1", "value": 5 }], "output": [{ "name": "outputname1", "type": "integer", "description": "description for outputname1", "value": null }], "optionalParameters": { "freeFormParams": "for the author to use", "param1":"value1" } }, "producedBy": "[, ]", "kpi": [{ "name": "error rate", "value": 0.9 }] } ``` ## Obtaining the Metadata or Payload of a Specific Version of a Model This request can be used to download either the metadata or the payload of the specific version of a model. If the `Content-Type` header is specified as `application/octet-stream`, the service returns the specified version's payload. In this sample the `Content-Type` is specified as `application/json`, for which the service returns the metadata of the model's specified version. ```http GET /api/modelmanagement/v3/models//versions/ HTTP/1.1 Content-Type: `application/json` ``` Response: ```json { "id": "", "number": 1.0, "expirationDate": "2019-10-01T12:00:00.001", "author": "user@siemens.com", "creationDate": "2018-10-01T12:00:00.001", "dependencies": [{ "name": "sklearn-theano", "type": "Python", "version": "1.7" }], "io": { "consumes": "CSV", "input": [{ "name": "variablename1", "type": "integer", "description": "description for variablename1", "value": 5 }], "output": [{ "name": "outputname1", "type": "integer", "description": "description for outputname1", "value": null }], "optionalParameters": { "freeFormParams": "for the author to use", "param1":"value1" } }, "producedBy": [, ], "kpi": [{ "name": "error rate", "value": 0.9 }] } ``` ## Deleting a Model When deleting a model, all of its related versions are deleted successively. Versions are deleted from smallest to largest version number. After deleting all associated metadata and payload files, the model itself is deleted. ```http DELETE /api/modelmanagement/v3/models/` ``` Response: ```http Status: 204 - No content ``` ## Deleting the Last Version of a Model ```http DELETE /api/modelmanagement/v3/models//versions/last ``` Response: ```http Status: 204 - No content ``` ## Deleting a Specific Version of a Model Both the model ID and the version ID must be specified for this service. ```http DELETE /api/modelmanagement/v3/models//versions/ ``` Response: ```http Status: 204 - No content ``` ## Managing Docker images Through this API we are now providing Docker image management support. All the operations available for regular (file based) payloads are also available for Docker images (`type='Docker Image`). ## Uploading a Model with a previously uploaded Docker image Once the Docker image has been uploaded the repository that hosts the image needs its metadata associated with an AMM entity. For that, the following needs to be accomplished: ```http POST /api/modelmanagement/v3/models ``` Provide the following content for the `metadata` key of the `form-data`: ```json { "name": "My Docker Model 1", "description": "Docker image 1", "type": "Docker Image", "lastVersion": { "expirationDate": "2025-10-01T12:00:00.001" } } ``` In this request, the `number` attribute is not required for `type="Docker Image"` since this is extracted from the URI's tag. The provided image tag will be filling the `number` attribute automatically, hence why this is not required. The `uri` key of the `form-data` must contain the URI repository of the Docker image, plus the desired tag; if no tag is specified, then `latest` is being assumed. The repository URI form needs to match the `URI:tag` format. If `latest` tag or the specified tag does not exist in the repository, the request will fail. Response: ```json { "id": "", "name": "My Docker Model 1", "description": "Docker image 1", "type": "Docker Image", "lastVersion": { "id": "", "uri": ":<>" } } ``` ## Uploading a Version for a previously uploaded Docker image You can upload Docker images at your own pace, and define as many versions(tags) as you need. But in order to make use of that new image tag in AMM you need to associate its URI and tag with the Version metadata. For that, you need to follow identical steps like in the Version creation, but you also need to specify the Docker image URI and tag. ```http POST /api/modelmanagement/v3/models//versions ``` Provide the following Version content for the `metadata` key of the `form-data`: ```json { "expirationDate": "2025-10-01T12:00:00.001" } ``` In this request, the `number` attribute is not required for `type="Docker Image"` models, since this is extracted from the URI's tag. The provided image tag will be filling the `number` attribute automatically, hence why this is not required. Also, the `uri` key of the `form-data` must contain the URI repository of the Docker image, plus the desired tag; if no tag is specified, then `latest` is being assumed. The repository URI form needs to match the `URI:tag` format. If `latest` tag or the specified tag does not exist in the repository, then the request will fail. Response: ```json { "id": "", "number": "", "expirationDate": "2024-10-01T12:00:00.001", "creationDate": "2021-01-07T09:49:47.704", "lastModified": "2021-01-07T09:49:47.704", "author": "myuser@company.com", "uri": ":" } ``` ### Obtain temporary tokens for Docker operations Simply perform a GET request on this endpoint: ```http GET /api/modelmanagement/v3/repositories/getAccessToken HTTP/1.1 Content-Type: `application/json` ``` and the response is in the following form: ```json { "credentials": { "user": "", "password": "", "registry": "" }, "uri": "", "providerCredentials": { "accessKey": "", "secret": "", "sessionToken": "" }, "expirationTime": "2019-01-07T07:52:34.000" } ``` ### Uploading a new docker image Using the temporary token for a Docker operation that can be obtained using `GET /api/modelmanagement/v3/repositories/getAccessToken`, you can upload a Docker image to this AMM repository by following these steps: 1. Log in with your Docker CLI by `docker login -u -p ` 1. Tag your local image with the provided repository URI `docker tag localimage:vTag1.0 :` 1. Upload the docker image in the repository\ `docker push :myRemoteTag1.0` and wait for the upload to complete 1. Associate AMM metadata with the newly image by creating a regular AMM model where you also specify the new repository as shown in [Uploading a Model with a previously uploaded Docker image](#uploading-a-model-with-a-previously-uploaded-docker-image) # Agent Management Service – API Overview ## Region Europe 1 [Download OpenAPI Specification](/insights-hub/api_specs/agentmanagement-v3-6-0-xcelerator.yaml) # Agent Management API - Usage quota and limits Agent Management APIs are restricted with following limits. | API Endpoint | Throttling/Restriction Criteria | XS | S | M | L | XL | | ------------- | ------------------------------- | --- | --- | --- | --- | --- | | Get agents | Requests in 30 Seconds | 10 | 15 | 15 | 15 | 15 | | Get agents | Requests in 3 Minutes | 40 | 100 | 100 | 100 | 100 | | Delete agents | Requests in 30 Seconds | 10 | 10 | 10 | 10 | 10 | | Delete agents | Requests in 3 Minutes | 20 | 30 | 30 | 30 | 30 | Note Rate Limits are defined considering the valid usage metrics based on agents' count. Ensure to check the rate limits before implementing APIs. # Agent Management Service ## Idea The Agent Management Service is typically used by application developers or machine builders (OEMs). It provides APIs to onboard, offboard, update and delete agents. Connectivity functions are used to enable communication with Industrial IoT. ## Access For accessing this service, you need to have the respective roles listed in [Agent Management roles and scopes](https://documentation.mindsphere.io/MindSphere/concepts/concept-roles-scopes.html#agent-management). Users can only interact with agents onboarded within their environment. ## Basics ### Agents Agents are the primary actors within the environment. For example, they upload data, retrieve events, change their configuration etc. The very first step for using Industrial IoT APIs is to create an agent in Insights Hub. When the agent is created, an initial access token (IAT) is generated. The IAT is a JSON Web Token (JWT) that holds various information about the agent and must be downloaded to the agent. The IAT is valid for one week (7 days). It required to [onboard](#onboarding) the agent to Insights Hub. Insights Hub checks the signature of the IAT to validate the agent. After onboarding, the agent is authorized and authenticated to communicate with Insights Hub. Agents provide their credentials using one of the following security profiles: - SHARED_SECRET - RSA_3072 #### SHARED_SECRET Security Profile Insights Hub creates a secret for agents with this security profile and stores it in its persistent storage. This secret is returned to the agent in the onboarding response. #### RSA_3072 Security Profile Agents with this security profile first send their public key to Insights Hub when onboarding. Insights Hub stores the public key in its persistent storage. ### Data Source Configuration Insights Hub needs a data source configuration for interpreting the data it receives from an agent. Without this configuration, Insights Hub cannot understand the data. The data source configuration contains data sources and data points. Data sources are logical groups, for example, a sensor or a machine, which contains one or more measurable data points, like temperature or pressure. When an agent is first created, its data source configuration is empty and must be updated using this service. ### Onboarding Agents are onboarded using the `Register` endpoints of this service. The onboarding process follows the OAuth 2.0 Authorization protocol (RFC 6749): 1. The agent retrieves the Initial Access Token (IAT) from its boarding configuration. 1. It sends an onboarding request singed with the IAT according to its security profile. 1. If onboarding is successful, Insights Hub responds with a Registration Access Token (RAT), which is valid indefinitely. Agent credentials have to be updated using the `/register/{id}` endpoint every 7 days, regardless of the security profile. In the update process, the agent has to provide the RAT instead of IAT to update its credentials, otherwise the process is the same as for initial registration. Agents are offboarded using the `/agents/{id}/boarding/offboard` endpoint. When an agent is offboarded, a new IAT is generated by Insights Hub, which allows to onboard the agent again. ### Access Token Agents need an access token to consume Industrial IoT services. This is a time restricted JSON Web Token (JWT) that holds the agent's scopes (access rights) among other information. Agents must be onboarded before they can acquire an access token, which requires the following steps: 1. The agent creates a self-signed JWT, which holds information such as agent ID, environment name etc., and signs it with its shared secret or private key based on chosen security profile. 1. The agent sends an access token request with the self signed JWT to Insights Hub. 1. Insights Hub validates the signature of the JWT using the stored credentials of the agent. 1. If the self signed JWT is valid, Insights Hub responds with an access token. Note Token generation and grants comply to the rules stated by Oauth2.0 authorization framework. Note Access tokens are valid for one hour. After its expiration, agents need to acquire a new access token to continue using Industrial IoT services. Agent Access token will have the role `mdsp:core:DefaultAgent` which has the scopes listed in [Agent Access Token roles and scopes](https://documentation.mindsphere.io/MindSphere/concepts/concept-roles-scopes.html#agent-access-token). ## Limitations In order to give optimal performance, Agent Management API provides technical limits on API usage and resources which needs to be incorporated while using it. API technical limits are documented in the [Agent Management API - Usage quota and limits](api-agentmanagement-apiratelimits.html) chapter. The following are the technical limits for resource usage: | Resource | XS | S | M | L | XL | | ---------------- | --- | --- | --- | --- | --- | | Number of agents | 50 | 200 | 300 | 400 | 500 | Note These limits are enforced on Capability Package based environments only. ## Features - Create, edit, remove agents - Onboard and offboard agents - Define an agent's data sources - Acquire access tokens to consume Insights Hub services ## Example Scenario The application developer of a brewery wants to programmatically onboard and offboard MindConnect devices connected to the production lines. The developer uses the Agent Management Service to register and offboard the desired devices. ## Related Links - [Onboarding an agent](https://documentation.mindsphere.io/MindSphere/howto/howto-agent-onboard.html) - [Acquiring an agent access token](https://documentation.mindsphere.io/MindSphere/howto/howto-agent-access-token.html) - [Uploading agent data](https://documentation.mindsphere.io/MindSphere/howto/howto-agent-upload-data.html) - [MindConnect Service](../connectivity-mindconnect/api-mindconnect-overview.html) - [Authentication & Authorization](https://documentation.mindsphere.io/MindSphere/concepts/concept-authentication.html) # Agent Management Async API – API Specification This is an API that uses Async API specification. You can download the Agent Management Async API specification. [Download Specification](/insights-hub/api_specs/agentmanagement-async-v3-0-0-rancher.yaml) # Agent Management Async API – API Specification This is an API that uses Async API specification. You can download the Agent Management Async API specification. [Download Specification](/insights-hub/api_specs/agentmanagement-async-v3-0-0-eu1.yaml) # Agent Management Async API – API Overview ## Available API versions | Region | API version | | -------- | -------------------------------------------------------------- | | Europe 1 | [3.0.0](api-agentmanagement-async-api-swagger-v3-0-0-eu1.html) | # Agent Management Async API ## Idea Agent Management Async API defines topics and message structures to request agent access tokens. The agent access token enables MQTT agents to perform other RESTful operations such as getting asset types, aspect types, large file upload, and writing to IDL. For further information about the Agent Management Async API, refer to the [Agent Management Async API specification](api-agentmanagement-async-api.html). Info The Agent Management Async API is currently available in region Europe 1. ## Access For accessing these APIs, you need to upload your CA Certificate. For more information, refer [Managing CA Certificates](https://documentation.mindsphere.io/MindSphere/howto/howto-managing-ca-certificates.html). The agent platforms using this API must support MQTT. ## Basics ### Agent Access Token Scopes Agent Access token has the role `mdsp:core:DefaultAgent` which has the scopes listed in [Agent Access Token roles and scopes](https://documentation.mindsphere.io/MindSphere/concepts/concept-roles-scopes.html#agent-access-token). ## Features The Agent Management exposes its Async API to MQTT agents for realizing the following tasks: - Request agent access token. - Receive the result of an agent access token request. ## Limitations - There are frequency limits per environment and per client for published and received messages. - Agent Access Token is valid for one hour. After its expiration, agents need to acquire a new agent access token to continue using Industrial IoT services. ## Example Scenario The MQTT agent needs to exchange large data files with Insights Hub. Sending this through the MQTT broker increases the load for active messages as well as imposes a limitation on the size of the file. The broker and the protocol itself have message limits that are far lower than some use cases. This can be reduced by allowing the agents to directly access the MindConnect API. ## Related Links - [Acquiring an Agent Access Token via MindConnect MQTT Agents](https://documentation.mindsphere.io/MindSphere/howto/howto-acquiring-an-agent-access-token-via-mindconnect-mqtt-agents.html) - [MindConnect API](../connectivity-mindconnect/api-mindconnect-overview.html) - [MindConnect MQTT API](../connectivity-mindconnect-mqtt/api-mindconnect-mqtt-overview.html) - [MindConnect MQTT Broker](https://documentation.mindsphere.io/MindSphere/concepts/concept-mindconnect-mqtt-broker.html) # Asset Modeler Async API Service – API Overview ## Available API versions | Region | API version | | -------- | ----------------------------------------------------------------------------------------------------------------------- | | Europe 1 | [3.1.0](api-assetmodeler-async-api-swagger-v3-1-0-eu1.html) [4.0.0](api-assetmodeler-async-api-swagger-v4-0-0-eu1.html) | # Asset Modeler Async API Service – API Specification This is an API that uses Async API specification. You can download the Asset Modeler Async API specification. [Download Specification](/insights-hub/api_specs/assetmodeler-async-v3-0-0-rancher.yaml) # Asset Modeler Async API Service – API Specification This is an API that uses Async API specification. You can download the Asset Modeler Async API specification. [Download Specification](/insights-hub/api_specs/assetmodeler-async-v3-1-0-eu1.yaml) # Asset Modeler Async API Service – API Specification This is an API that uses Async API specification. You can download the Asset Modeler Async API specification. [Download Specification](/insights-hub/api_specs/assetmodeler-async-v4-0-0-eu1.yaml) # Asset Modeler Async API Service – API Specification This is an API that uses Async API specification. You can download the Asset Modeler Async API specification. [Download Specification](/insights-hub/api_specs/assetmodeler-async-v4-0-0-rancher.yaml) # Asset Modeler Async API Service ## Idea One of the significant hurdles after connecting your agents to the cloud is to represent their data points in the asset model accurately along with desired asset structure and its hierarchy. Allowing the user to perform this operation manually can often lead to a subpar experience due to its susceptibility to errors and the extensive time it demands. It requires a skilled person to spend their quality time. This becomes very expensive even if you want to maintain a simple asset hierarchy along with countable number of data points.\ It would be helpful if the agent could automatically construct and maintain the entire asset model and its hierarchy for the users. This allows the users to quickly realize the value of the data collected by your agent. By using Asset Modeler services, your agent can efficiently create and maintain the entire asset model from your agent. The user then can refrain from manually creating and maintaining the asset model. This will enable the user to focus on the data gathered by the agents. You can use Asset Modeler Async services to create and maintain the asset model. These services provide you with topics and message structures for this purpose. It can also be used for maintaining a suitable asset hierarchy. ## Features Asset Modeler services can be used to create the asset model in either of the following ways: - Create all elements of the asset model - Create only portions of the asset model as per the requirement Asset Modeler Async services allows the agents to perform the following operations: - Create an entire asset model by describing aspect types, asset types, asset instances and data point mappings. In this case, every element of the asset model is created by the Asset Modeler services. - Reuse the portions of asset model to create a new one or update an existing asset model. In this case, only the new definition of the asset model is created. - Instantiate already defined asset model to quickly create its components - Update only portion of the model to modify the model that is already existing - Delete the portion of the model that is no longer needed - Subscribe to the changes to the model from Insights Hub either partially or fully For further information about the Asset Modeler Async API, refer to [Asset Modeler Async API Specification](api-assetmodeler-api.html). Info The Asset Modeler Async API Service is currently available in regions Europe 1 and Rancher. ## Access For accessing this API, you need to upload your CA Certificate. For more information, refer to [Manage MQTT Certificates](https://documentation.mindsphere.io/MindSphere/howto/howto-managing-ca-certificates.html). ## Basics ### Broker information The MQTT devices must authenticate the broker with the help of a certificate. You need to download the provided certificate and install it on your device. ### Environment You need an environment on Industrial IoT. Within this document, parameter `\$TENANT` is the name of your environment. ### ClientId The `clientId` represents the agent identification. MQTT Client of agents must be unique and exclusive. The `ClientId` is the key to the session stored in broker, hence exclusiveness is security relevant in a multi-tenant system. That `ClientId` is also associated with the thing that represents the client by setting thing's name to `ClientId`. Since `ClientId` is selected by connecting (registering) agent, it is possible to have conflicts in ids. To avoid the conflicts, client selection is restricted to the following format: `_` Then, such a relation is enforced: ` = = _` Info - Tenant name must be known prior to issuing device certificates. - Agents needs to be configured with environment name and certificate's CN. - Customer is responsible to assign CNs uniquely for each device. ### Models Models are templates describing asset/aspect types, asset instances, asset hierarchy and mappings from data point ids to variables in aspects. This operation creates an own asset model by describing aspect types, asset types, asset instances and data point mappings. ### Instantiations This operation creates a new model with a given asset model in the body. An instantiation job takes a model and creates the items described in the model by creating types, instances, and mappings. A model can be instantiated many times. Models and instantiations are loosely coupled. A change to model instantiation does not affect the model. Also, a change in the model does not affect any completed instantiations as well.\ For example, if a model is deleted, existing instantiations are not affected. Or, moving an asset created by an instantiation, does not affect the model. ### Instantiation Progress This operation is used to get instantiation job results for a given instantiation request. ### Model Creation Result Receive model creation results for a previous model request. ### Instantiation Progress Result Receive instantiation job results for a previous instantiation request. ## Limitations - In Asset Modeler V3, it is possible to upload models to Insights Hub. However, retrieval of these models is not supported, and neither deletion nor updating of resources is permitted. The model is not updated after a change made through Asset Manager. - There are frequency limits per environment and per agent for published and received messages. - Maximum size of the body is 128 KB. - One-to-one relation exists between the client and the asset model for Asset Modeler V4. - The enablement of Asset Modeler V4 API may take up to 15 minutes after the device is connected. ## Example Scenario The manager of a wind farm wants to collect sensor data of a wind turbine and collect the data of Rotation Speed, Air Density, Turbulence, Pulse Duration, Pulse Log, and Stator Voltage data. This data is required to reduce operational risks and improve efficiency. The developer implements a publisher to create the `WindTurbinePowerGeneratorModel`, WindTurbine data collection asset instances and data mappings via Asset Modeler Async API. ## Related Links - [MindConnect Async API](../connectivity-mindconnect-async/api-mindconnect-async-overview.html) - [MindConnect MQTT Broker](https://documentation.mindsphere.io/MindSphere/concepts/concept-mindconnect-mqtt-broker.html) - [Manage MQTT Certificates](https://documentation.mindsphere.io/MindSphere/howto/howto-managing-ca-certificates.html) - [Asset Modeler How To](https://documentation.mindsphere.io/MindSphere/howto/howto-mindconnect-mqtt.html) - [Creating V4 Data Model from MQTT Agent](https://documentation.mindsphere.io/MindSphere/howto/howto-create-v4-data-model-mqtt-agent.html) - [Creating V3 Data Model from MQTT Agent](https://documentation.mindsphere.io/MindSphere/howto/howto-create-data-model-mqtt-agent.html) - [Transition to V4 Model](https://documentation.mindsphere.io/MindSphere/howto/howto-transition-to-v4-model.html) # Asset Modeler Async API Service – Samples The code examples given in this section are json representations of the Asset Model including the mappings. ## Topic Structure MQTT Broker offers topics described below: ``/``/``/``/`_`/``\ ``: [tc] - tc: tenant-client topic structure - ``: environment id - ``: Unique client id - ``: [i|o] - i: inbound. Clients can subscribe - o: outbound. Clients can publish - ``: Registered application topic name - ``: Version of application - ``: App specific sub topics PUBLISH: `tc///o/amo_v3/m` Models are templates describing asset/aspect types, asset instances, asset hierarchy and mappings from data point ids to variables in aspects. SUBSCRIBE : `tc///i/amo_v3/ms` Receive model creation results for a previous model request. PUBLISH: `tc///o/amo_v3/i` Instantiations are the realization of models. An instantiation job takes a model and creates the items described in the model by creating types, instances and mappings. A model can be instantiated many times. SUBSCRIBE : `tc///i/amo_v3/ip` Receive instantiation job results for a previous instantiation request. ## Create own Asset model This operation creates own asset model by describing aspect types, asset types, asset instances and data point mappings. **Asset Model message:** ```json { "id": "72609761sdv245248e126814c2dd83a27f", "data": { "externalId": "SpaceShip", "typeModel": { "aspectTypes": [{ "id": "connint6.wing", "name": "${aspectTypeName}", "category": "static", "scope": "private", "variables": [{ "name": "temperature", "dataType": "STRING", "unit": "C/F", "searchable": true, "length": 5, "qualityCode": true }], "description": "wing aspect type description", "referenceId": "287adc1a086840e0a6721dfd1170e97c" }], "assetTypes": [{ "id": "connint6.spaceship", "name": "wingAssetTypeName", "parentTypeId": "core.basicasset", "aspects": [{ "name": "wingAspect", "aspectTypeId": "connint6.wing" }], "description": "Hyperspace jump capable space ship", "instantiable": true, "scope": "private", "referenceId": "82a4cc2a69cc42af80c1c6cf5dbefde5" }] }, "instanceModel": { "assets": [{ "referenceId": "wingAssetReference", "parentReferenceId": "root", "typeId": "connint6.spaceship", "name": "wingAsset", "description": "The ship of Han Solo and Chewbacca" }] }, "mappingModel": { "mappings": [{ "dataPointId": "dp01", "assetReferenceId": "wingAssetReference", "aspectName": "wingAspect", "variableName": "temperature", "referenceId": "19e9048e78f540e7a9ba25e1249fea9b" }] } } } ``` ## Receive Model Creation Status This operation can be used to receive model creation status for the previous model request. ```json { "id": "01FQC37FCCASSFT7M0D6W5EB8P", "correlationId": "01FQC37FCBG1DWMTEC4DYCZ327", "requestId": "72609761sdv245248e126814c2dd83a27f", "data": { "id": "01FQC37HX2KGP4DR0HNVR3K5B2", "externalId": "SpaceShip", "status": "Success" } } ``` ## Create Model Instantiations This operation creates a new instance with a given asset model in the body. **Instantiate model message:** ```json { "id": "bb9b9d8d6193ddsss45a5w2s7e848fe28c", "data": { "modelExternalId": "SpaceShip", "parameterization": { "values": [{ "name": "aspectTypeName", "value": "wingAspectTypeName" }] } } } ``` ## Receive Instantiation Job results This operation can be used to receive instantiation job results for previous instantiation request. Instantiation Job result message: ### Message 1 ```json { "id": "01FQC37RDJ54CAM75RWNAQV781", "correlationId": "01FQC37NKDHM4774NB1R1H7PST", "data": { "id": "01FQC37NVVFZWET43NSHYZJV4B", "modelId": "01FQC37HX2KGP4DR0HNVR3K5B2", "modelExternalId": "SpaceShip", "message": "25% completed.", "status": "InProgress" }, "requestId": "bb9b9d8d6193ddsss45a5w2s7e848fe28c" } ``` ### Message 2 ```json { "id": "01FQC37RDJ54CAM75RWNAQV781", "correlationId": "01FQC37NKDHM4774NB1R1H7PST", "data": { "id": "01FQC37NVVFZWET43NSHYZJV4B", "modelId": "01FQC37HX2KGP4DR0HNVR3K5B2", "modelExternalId": "SpaceShip", "message": "50% completed.", "status": "InProgress" }, "requestId": "bb9b9d8d6193ddsss45a5w2s7e848fe28c" } ``` ### Message 3 ```json { "id": "01FQC37RDJ54CAM75RWNAQV781", "correlationId": "01FQC37NKDHM4774NB1R1H7PST", "data": { "id": "01FQC37NVVFZWET43NSHYZJV4B", "modelId": "01FQC37HX2KGP4DR0HNVR3K5B2", "modelExternalId": "SpaceShip", "message": "75% completed.", "status": "InProgress" }, "requestId": "bb9b9d8d6193ddsss45a5w2s7e848fe28c" } ``` ### Message 4 ```json { "id": "01FQC37RDJ54CAM75RWNAQV781", "correlationId": "01FQC37NKDHM4774NB1R1H7PST", "data": { "id": "01FQC37NVVFZWET43NSHYZJV4B", "modelId": "01FQC37HX2KGP4DR0HNVR3K5B2", "modelExternalId": "SpaceShip", "message": "100% completed.", "status": "Success" }, "requestId": "bb9b9d8d6193ddsss45a5w2s7e848fe28c" } ``` # Commanding Feature – API Overview ## Region Europe 1 [Download OpenAPI Specification](/insights-hub/api_specs/commanding-v3-3-0-xcelerator.yaml) # Commanding Feature -API Overview ## Available Async API Versions | Region | Async API version | | -------- | ------------------------------------------------------- | | Europe 1 | [3.1.1](api-commanding-asyncapi-swagger-3-1-1-eu1.html) | ## Available Sync API Versions | Region | Sync API version | | -------- | -------------------------------------------------- | | Europe 1 | [3.3.0](api-commanding-api-swagger-3-3-0-eu1.html) | # Commanding Feature – API Specification This is an API that uses Async API specification. You can download the Commanding API specification. [Download Specification](/insights-hub/api_specs/commanding-async-v3-1-1-eu1.yaml) # Commanding Feature – API Specification This is an API that uses Async API specification. You can download the Commanding API specification. [Download Specification](/insights-hub/api_specs/commanding-async-v3-1-1-rancher.yaml) # Commanding Feature ## Idea Commanding feature provides functionality for applications to send data up to 20 MindConnect MQTT agents. The data could be a command and/or data associated with the command sent to the agents. This feature provides RESTful APIs for the applications to directly send the command and associated data to the agents. ## Access For accessing the commanding feature, you need to have the respective roles listed in [Commanding Feature roles and scopes](https://documentation.mindsphere.io/MindSphere/concepts/concept-roles-scopes.html#commanding-api). ## Basics A `core.mcmqtt` agent must be onboarded to access the commanding feature. For more information, refer to [Onboarding MindConnect MQTT Agent](https://documentation.mindsphere.io/MindSphere/howto/howto-onboard-mindconnect-mqtt.html). Once the agent is onboarded, the agent needs to subscribe to a corresponding MQTT topic to receive commands. For instructions to configure MQTT agents, refer [Commanding Feature Async API specifications](api-commanding-api.html). In order to respond to a command received, the agent must publish to a predefined topic. For information about the predefined topics, refer to [Commanding Feature Sync API specifications](api-commanding-api.html). ## Features The Commanding feature exposes its API for applications and services to realize following tasks: - Create delivery jobs - List all delivery jobs - Get, Delete a delivery job - Get all commands related to a delivery job ## Limitations - The maximum number of agents to which a command can be published is 20. - The command data size is restricted to 10 kB in region Europe 1 and 4 kB in Private Cloud. ## Example Scenario The application developer of a brewery wants to send commands to certain group of MQTT agents in the brewery's production line. ## Related Links - [Onboarding MindConnect MQTT Agent](https://documentation.mindsphere.io/MindSphere/howto/howto-onboard-mindconnect-mqtt.html) - [Managing CA Certificates](https://documentation.mindsphere.io/MindSphere/howto/howto-managing-ca-certificates-using-api.html) - [Creating Data Model from MQTT Agent](https://documentation.mindsphere.io/MindSphere/howto/howto-create-data-model-mqtt-agent.html) # Commanding Feature – Samples for Publishing MQTT commands to agents Commanding Feature is a service that provides the APIs to manage delivery jobs to send commands to MQTT agents for execution. The examples below exemplify how to use the provided endpoints: ## Prerequisites - Onboard a `core.mcmqtt` agent. For more information, refer to [Onboarding MindConnect MQTT Agent](https://documentation.mindsphere.io/MindSphere/howto/howto-onboard-mindconnect-mqtt.html). - Agent must subscribe to a corresponding MQTT topic. For instructions to configure MQTT agents, refer [Commanding Feature Async API specifications](api-commanding-api.html). ## Creating a delivery job The Commanding feature provides functionality for sending MQTT command to a list of agents for execution. The following validations are performed when a delivery job is created: - The provided list of agents must be valid. - Maximum number of agents can be 20 in a delivery job. - Data size is restricted to 4KB. A command is sent using the following endpoint: ```http POST/deliveryJobs ``` Sample Request: ```json { "name": "firmware update", "clientIds": [ "mytenant_201541ad38a4495a96e2021c762b647f" ], "data": { "additionalProp1": "string", "additionalProp2": "string", "additionalProp3": "string" }, "createdBy": "user@mailId.com" } ``` Sample Response: ```json { "id": "21349765fe45652c8e126814c283a114", "name": "firmware update", "clientIds": [ "mytenant_201541ad38a4495a96e2021c762b647f" ], "tenantId": "mytenant", "status": "EXECUTING", "data": { "additionalProp1": "string", "additionalProp2": "string", "additionalProp3": "string" }, "createdAt": "2011-08-12T20:17:46.384Z", "createdBy": "user@mailId.com" } ``` The ID generated `tenantId` is referenced further to get, delete a delivery job and get all commands for a given delivery job. For more information, refer [Commanding Feature Sync API specifications](api-commanding-api.html). ## Listing all delivery jobs Use the following endpoint to list all delivery Jobs: ```http GET/deliveryJobs ``` Sample Response: ```json { "_embedded": { "deliveryJobs": [ { "id": "21349765fe45652c8e126814c283a114", "name": "firmware update", "status": "EXECUTING", "createdAt": "2011-08-12T20:17:46.384Z" } ] }, "page": { "size": 0, "totalElements": 0, "totalPages": 0, "number": 0 }, "_links": { "self": { "href": "string" }, "first": { "href": "string" }, "prev": { "href": "string" }, "next": { "href": "string" }, "last": { "href": "string" } } } ``` For more information, refer [Commanding Feature Sync API specifications](api-commanding-api.html). ## Managing delivery jobs Here, `tenantId`, the id generated while [Creating a delivery job](#creating-a-delivery-job) is used. - Use the following endpoint to get a delivery job: ```http GET​/deliveryJobs​/{id} ``` - Use the following endpoint to delete a delivery job: ```http DELETE/deliveryJobs​/{id} ``` - Use the following endpoint to get the all the commands for a given delivery job: ```http GET/deliveryJobs​/{id}​/commands ``` - Use the following endpoint to get the command associated with a delivery job by command id: ```http GET/deliveryJobs/{id}/commands/{commandId} ``` For more information, refer [Commanding Feature Sync API specifications](api-commanding-api.html). # MQTT Commanding Feature – Samples for subscribing to an MQTT topic Commanding Feature Async API defines topics and message structures to subscribe MQTT commands to be executed by the agents and publish the command execution status. For more information, refer [Commanding Feature Async API specifications](api-commanding-api.html). ## Subscribing Commands This operation is used to receive the command to be executed by the agent. | Region | Operation | | -------- | ------------------------------------- | | Europe 1 | `tc/{tenantId}/{clientId}/i/cmd_v3/c` | Payload example: ```json { "id": "99609765fed2452c8e126814c283a27f", "data": { "jobId": "21349765fe45652c8e126814c283a114", "createdAt": "string", "payload": { "property1": "string", "property2": "string" } } } ``` ## Publishing Command Execution Status This operation is used to publish the command execution status and progress. | Region | Operation | | -------- | ------------------------------------- | | Europe 1 | `tc/{tenantId}/{clientId}/o/cmd_v3/u` | Payload example: ```json { "id": "11609765fed2452c8e126814c283a289", "requestId": "99609765fed2452c8e126814c283a27f", "data": { "timestamp": "2011-08-12T20:17:46.384Z", "jobId": "21349765fe45652c8e126814c283a114", "status": "EXECUTING", "response": { "property1": "string", "property2": "string" } } } ``` # Custom Data Services API – API Overview ## Region Europe 1 [Download OpenAPI Specification](/insights-hub/api_specs/bringyourowndataformat-v1-0-1-xcelerator.yaml) # Custom Data Services API ## Idea The Custom Data Services API enables users to register and manage custom topic mappings for data integration within Insights Hub. This API provides a comprehensive set of endpoints for creating, retrieving, updating and deleting custom topic registrations and their associated mappings. ## Access For accessing this service, you need to have the respective roles listed in [MindConnect API roles and scopes](https://documentation.mindsphere.io/MindSphere/concepts/concept-roles-scopes.html#mind-connect-api). ## Basics ### Custom Topic Registrations The core functionality of this API is to manage custom topic registrations for data integration within Insights Hub. You can create, retrieve, update, and delete custom topic mappings and their associated configurations. ### Mapping Information The mapping information represents the details of a custom data mapping, which includes: - Asset ID - Aspect Name - Variable Name - Value Expression - Timestamp Expression - Time Converter settings ### Time Converter Options This API supports multiple time format conversions, such as Epoch, Unix Timestamp, RFC2822, RFC822, RFC3339, MySQL Timestamp, Human Readable Format, and JavaScript Date Format. You can specify the appropriate time converter for their data requirements. ### Optimistic Locking The API uses the ETag mechanism for optimistic locking to prevent concurrent modifications. ## Limitations In order to give optimal performance, Custom Data Services API provides technical limits on API usage and resources which needs to be incorporated while using it. The following are the technical limits for resource usage: | Resource | Public Cloud | Local Private Cloud (LPC) | | ----------------------------- | ------------ | ------------------------- | | Max topics allowed per agent | 5 | 10 | | Max topics allowed per tenant | 50 | 3000 | | Max topic hierarchy allowed | 7 | 15 | Note If the converter option is set as ISO8601 and the timestamp ingested has a precision of more than 3 digits (in millisecond unit) for example, 2025-03-28T07:00:14.123456789Z, the timestamp forwarded to the Insights Hub Monitor is 2025-03-28T07:00:14:123Z. This is because the time series handles timestamps only with upto 3 millisecond of precision. ## Features The Custom Data Service exposes its API for realizing the following tasks: - Create new topic mappings - Retrieve, update and delete the existing custom data mappings - Manage mapping rules and expressions - Support for multiple time format conversions ## Example Scenario A manufacturing company wants to integrate its own sensor data into the Insights Hub platform to leverage the advanced analytics and monitoring capabilities. By leveraging the Custom Data Services API, this manufacturing company can seamlessly integrate its proprietary data sources with the Insights Hub. ## Related Links - [Uploading Agent Data](https://documentation.mindsphere.io/MindSphere/howto/howto-agent-upload-data.html) - [Asset Management](../advanced-assetmanagement/api-assetmanagement-overview.html) # MindConnect API – API Overview ## Region Europe 1 [Download OpenAPI Specification](/insights-hub/api_specs/mindconnect-v3-7-0-xcelerator.yaml) # MindConnect API - Usage quota and limits MindConnect APIs are restricted with following limits. | API Endpoint | Throttling/Restriction Criteria | XS | S | M | L | XL | | ---------------------------------------- | ------------------------------- | ---- | ---- | ---- | ---- | ---- | | Post dataPointMappings | Requests in 1 Hour | 1050 | 1050 | 1050 | 1050 | 1050 | | Get dataPointMappings | Requests in 1 Hour | 1050 | 1050 | 1050 | 1050 | 1050 | | Get dataPointMappings/{id} | Requests in 1 Hour | 1050 | 1050 | 1050 | 1050 | 1050 | | Delete dataPointMappings/{id} | Requests in 1 Hour | 1050 | 1050 | 1050 | 1050 | 1050 | | Post eventMappings | Requests in 1 Hour | 1050 | 1050 | 1050 | 1050 | 1050 | | Get eventMappings | Requests in 1 Hour | 1050 | 1050 | 1050 | 1050 | 1050 | | Get eventMappings/{id} | Requests in 1 Hour | 1050 | 1050 | 1050 | 1050 | 1050 | | Patch eventMappings/{id} | Requests in 1 Hour | 1050 | 1050 | 1050 | 1050 | 1050 | | Delete eventMappings/{id} | Requests in 1 Hour | 1050 | 1050 | 1050 | 1050 | 1050 | | Post diagnosticActivations | Requests in 1 Minute | 20 | 20 | 20 | 20 | 20 | | Get diagnosticActivations | Requests in 1 Minute | 20 | 20 | 20 | 20 | 20 | | Get diagnosticActivations/{id} | Requests in 1 Minute | 20 | 20 | 20 | 20 | 20 | | Get diagnosticActivations/{id}/messages | Requests in 1 Minute | 20 | 20 | 20 | 20 | 20 | | Put diagnosticActivations/{id} | Requests in 1 Minute | 20 | 20 | 20 | 20 | 20 | | Delete diagnosticActivations/{id} | Requests in 1 Minute | 20 | 20 | 20 | 20 | 20 | | Post recoverableRecords/{id}/replay | Requests in 1 Minute | 20 | 20 | 20 | 20 | 20 | | Get recoverableRecords/{id}/downloadLink | Requests in 1 Minute | 20 | 20 | 20 | 20 | 20 | | Delete recoverableRecords/{id} | Requests in 1 Minute | 20 | 20 | 20 | 20 | 20 | | Get recoverableRecords | Requests in 1 Minute | 20 | 20 | 20 | 20 | 20 | Note Rate Limits can cause discontinuity in the use case flow. Ensure to check the rate limits before implementing APIs. # MindConnect API ## Idea MindConnect Services provide APIs that enables shop floor devices to send data securely and reliably. It allows custom applications (agents) to collect and upload data which shall be stored and used by applications in the cloud. ## Access For accessing this service, you need to have the respective roles listed in [MindConnect API roles and scopes](https://documentation.mindsphere.io/MindSphere/concepts/concept-roles-scopes.html#mindconnect-api). Agents need a field-side network infrastructure to forward and route outbound HTTPs requests to the Internet. MindConnect supports multiple agent device classes, strong hardware platforms as well as resource constrained devices. All target agent platforms must comply to the following minimum requirements: - HTTP processing - TLS - JSON parsing - JSON Web Token (JWT) generation - HMAC generation (preferably SHA2 based hashing) Users with IP based filtering on their firewall can use the following two static IPs for whitelisting the data upload traffic: ```json 75.2.111.226 99.83.250.213 ``` Enabling these two IP addresses is sufficient for agents and clients accessing `*.eu1.mindsphere.io`. If the agent or client is using certificate revocation list URLs, these have to be whitelisted in addition. However, this does not cover the interactive login process by Insights Hub/Siemens ID credentials and native access to IDL (AWS S3). A SSL implementation may want to check the revocation list of Certificate Authorities. If the SSL implementation is trying to get this certificate revocation list at runtime, it needs access to an external URL. It is used for distributing revoked CA certificates by the CA. However, it is more often done by operation system or java updates. ## Basics ### Data Source Configuration A data source configuration is needed for interpreting the data it receives from an agent. This configuration contains data sources and data points. Data sources are logical groups, for example, a sensor or a machine, which contains one or more measurable data points, like temperature or pressure. The data source configuration is defined using the [Agent Management Service](../connectivity-agentmanagement/api-agentmanagement-overview.html). For more information, refer to [Creating a Data Source Configuration](https://documentation.mindsphere.io/MindSphere/howto/howto-agent-upload-data.html#creating-a-data-source-configuration). ### Data Point Mapping Data point mapping is required for storing the data it receives from an agent. This maps the data points from the data source configuration to properties of the digital entity, that represents the agent. When Insights Hub receives data from an agent, it looks up which property the data point is mapped to and stores the data there. Use the MindConnect Service for defining the data point mapping. For more information, refer to [Creating a Data Point Mapping](https://documentation.mindsphere.io/MindSphere/howto/howto-agent-upload-data.html#creating-a-data-point-mapping). ### Event Mapping Events originating from the device in the field are sent to Insights Hub using MindConnect APIs. These events are stored in the corresponding agent asset. For example, if a field device is connected to Insights Hub over MCLIB agent, all the events originated from the field device is stored in the MCLIB (core.mclib type) agent. By using event mapping APIs from the agent, these events can now be mapped to appropriate asset in Insights Hub. The API allows you to define mapping criteria. For example, if the event type field source contains “MyMachine”, then map the events to the asset (asset id field in the API). Once the mapping is performed, every time the event of the selected type reaches the agent, the event is automatically stored against the asset. If otherwise, the event remains in the agent itself. - If there are no event mappings matching an event, that event will be stored in the agent asset. - Multiple mappings can match for an event uploaded. In such a case, all matching mappings will be applied for the event. - An asset can have multiple event mappings. - Maximum of 50 event mappings can be created per agent. - Maximum of 5 event mappings can be created per agent from an event type. Use the MindConnect Service for defining the event mapping. For more information, refer to [Creating Event Mapping](https://documentation.mindsphere.io/MindSphere/howto/howto-agent-upload-data.html#creating-event-mapping). ### Data Upload The MindConnect API allows agents to upload their data to Industrial IoT. This data can be one of the following types: - Time Series - File - Event The format conforms to a subset of the HTTP multipart specification, but only permits nesting of 2 levels. For more information, refer to [Uploading Agent Data](https://documentation.mindsphere.io/MindSphere/howto/howto-agent-upload-data.html#uploading-data). ### Standard data types The MindConnect Service uses standard data types, which allow Industrial IoT to automatically process the data without additional configuration or coding. This means: - The API defines how standard data types are transmitted, e.g. how metadata and production data need to be formatted as HTTPs payloads. - Standard data types are automatically parsed and the information is stored to (virtual) assets. - For each of the standard data types, there is a pre-configured mass data storage available. - Data of standard types can be accessed and queried in a standardized way by applications and analytical tools. The following standard data types for production data are supported: - **Time Series**\ Time Series are data point values that change constantly over time, for example, values from analog sensors like a temperature sensor. This also applies to any other measured values that have an associated timestamp. - **Events**\ Events are based on machine events, for example, emergency stops or machine failures. However, this mechanism can also be used to upload custom notifications, in instances where you do on-site threshold monitoring and want to report a broken threshold. - **Files**\ Files of up to 10 MB can be uploaded per exchange call. The files are attached to the corresponding (virtual) asset, e.g. device log files or complex sensor structures. Files that are uploaded can be referenced by the parent (virtual) asset. The content of these files is not parsed. It requires custom applications or analytical tools to interpret and visualize the data. - **Data Models**\ Data models describe the agent-side asset hierarchy and configuration including measurement points. ## Limitations In order to give optimal performance, MindConnect API provides technical limits on API usage and resources which needs to be incorporated while using it. API technical limits are documented in the [MindConnect API - Usage quota and limits](api-mindconnect-apiratelimits.html) section. The following are the technical limits for resource usage: | Resource | XS | S | M | L | XL | | --------------------------------------------------------------- | --- | --- | --- | --- | --- | | Max event mapping count per agent | 50 | 50 | 50 | 50 | 50 | | Max event mapping count per agent from an event type | 5 | 5 | 5 | 5 | 5 | | Number of aspect variables to which one datapoint can be mapped | 5 | 5 | 10 | 10 | 20 | Note These limits are enforced on Capability Package based environments only. ## Features The MindConnect Service exposes its API to agents for realizing the following tasks: - Upload time series - Upload files - Describe and upload asset data models - Upload data of custom data types for custom handling ## Example Scenario The manager of a wind farm wants to collect sensor data of a wind turbine. A developer implements a field application (agent) which collects the sensor data. The agent uses the MindConnect API for uploading the data to Industrial IoT. ## Related Links - [Uploading Agent Data](https://documentation.mindsphere.io/MindSphere/howto/howto-agent-upload-data.html) - [Asset Management](../advanced-assetmanagement/api-assetmanagement-overview.html) # Mindconnect API – Filtering ## General All endpoints with the parameter `filter` support basic filtering. Wildcards are not supported. Timestamps must be in ISO-8601 format as follows: `YYYY-MM-DDTHH:mm:ss.sssZ`, e.g.: `2018-10-04T07:58:49.369Z`. When a filter has no matches, the response will be empty. ### Filter functions #### Function `eq` (equals) ```json { "agentId": "3b27818ea09a46b48c7eb3fbd878349f" } ``` or ```json { "agentId": { "eq": "3b27818ea09a46b48c7eb3fbd878349f" } } ``` #### Function `in` The function `in` is very similar to `equals`, but instead matching to one value, it checks an array of values, and matches for any of them. ```json { "agentId": { "in": { "value": ["3b27818ea09a46b48c7eb3fbd878349f", "9b27818ea09a46b48c7eb3fbd878349f"] } } } ``` or ```json { "agentId": { "in": ["3b27818ea09a46b48c7eb3fbd878349f", "9b27818ea09a46b48c7eb3fbd878349f"] } } ``` #### Function `endsWith` ```json { "message": { "endsWith": "Successful." } } ``` #### Function `startsWith` ```json { "message": { "startsWith": "Dropped" } } ``` #### Function `contains` ```json { "message": { "contains": "Timeseries" } } ``` #### Function `before` ```json { "timestamp": { "before": "2018-01-01T00:00:00.00Z" } } ``` #### Function `after` ```json { "timestamp": { "after": "2018-01-01T00:00:00.00Z" } } ``` #### Function `between` Checks if the value of the field is between the given range Parentheses for inclusive date: [] Parentheses for exclusive date: () ```json { "timestamp": { "between": "[2018-01-01T00:00:00.00Z, 2018-01-31T00:00:00.00Z)" } } ``` ### Filter operations #### Operation `not` ```json { "not": { "message": { "startsWith": "Accepted" } } } ``` or ```json { "not": { "message":"Accepted" } } ``` #### Operation `or` ```json { "message": { "or": [ {"eq": "Data is Accepted"}, {"endsWith": "Accepted"} ] } } ``` or ```json { "or": { "message": { "eq": null }, "agentId": { "startsWith": "3b27818ea09a46b48c7eb3fbd878349f" } } } ``` #### Operation `and` ```json { "agentId":"3b27818ea09a46b48c7eb3fbd878349f", "message": null } ``` or ```json { "and":{ "message": { "eq": null }, "agentId": { "startsWith": "3b27818ea09a46b48c7eb3fbd878349f" } } } ``` # MindConnect Async API – Async API Specification This is an API that uses Async API specification. You can download the MindConnect Async API specification. [Download Specification](/insights-hub/api_specs/mindconnect-async-v3-0-0-rancher.yaml) # MindConnect Async API – API Specification This is an API that uses Async API specification. You can download the MindConnect Async API specification. [Download Specification](/insights-hub/api_specs/mindconnect-async-v3-1-0-eu1.yaml) # MindConnect Async API – API Overview ## Available API versions | Region | API version | | -------- | ---------------------------------------------------------- | | Europe 1 | [3.1.0](api-mindconnect-async-api-swagger-v3-1-0-eu1.html) | # MindConnect Async API ## Idea The MindConnect Async API provides topics and message structures to ingest data securely and reliably into Industrial IoT using MindConnect MQTT. The async APIs can be used for several purposes such as defining and initializing the models ([Asset Modeler Async API Service](../connectivity-assetmodeler/api-assetmodeler-overview.html)), exchanging time series data, sending events and so on. For further information about the MindConnect Async API, refer to the [MindConnect Async API specification](api-mindconnect-async-api.html). Info The MindConnect Async API Service is currently available in region Europe 1. ## Access For accessing these APIs, you need to upload your CA Certificate. For more information, refer [Managing CA Certificates](https://documentation.mindsphere.io/MindSphere/howto/howto-managing-ca-certificates-using-api.html). The agent platforms using this API must support MQTT. ## Basics ### Time Series Data Model Create the asset model and instantiate it before accessing this API, or else no data will be available on IoT Timeseries. For more information, refer to [Asset Modeler Async API Service](../connectivity-assetmodeler/api-assetmodeler-overview.html). ### Data Upload MindConnect Async APIs allows the agents to asynchronously upload the device data. The data can be of type: - Time Series Data - File - Events For instructions refer to [Sending Data from MindConnect MQTT Agent](https://documentation.mindsphere.io/MindSphere/howto/howto-send-data-from-mqtt-agent.html). ### Standard Data Types The MindConnect Service uses standard data types, which allows to automatic processing of data without additional configuration or coding. This means: - The API defines how standard data types are transmitted, for example, how metadata and production data need to be formatted as MQTT messages. - Standard data types are automatically parsed and the information is stored to (virtual) assets. - For each of the standard data types, there is a pre-configured mass data storage available. - Data of standard types can be accessed and queried in a standardized way by applications and analytical tools. The following standard data types for production data are supported: - **Time Series**\ Time Series are data point values that change constantly over time, for example, values from analog sensors like a temperature sensor. This also applies to any other measured values that have an associated timestamp. - **Events**\ Events are based on machine events, for example, emergency stops or machine failures. However, this mechanism can also be used to upload custom notifications, for example, if you do on-site threshold monitoring and want to report a broken threshold. - **Files**\ Files of up to 75 kB can be uploaded per publish request. The files are attached to the corresponding (virtual) asset, for example, device log files or complex sensor structures. Files that are uploaded can be referenced by the parent (virtual) asset. The content of these files is not parsed. It requires custom applications or analytical tools to interpret and visualize the data. ## Features The MindConnect Service exposes its Async API to agents for realizing the following tasks: - Upload time series - Upload files - Upload events ## Limitations - There are frequency limits per environment and per client for published and received messages. - The number of samplings in a message is limited. - The time series data and event payload for every publish request is limited to 128 KB. - Files of up to 75 KB can be uploaded per publish request. ## Related Links - [Asset Modeler Async API Service](../connectivity-assetmodeler/api-assetmodeler-overview.html) - [Sending Data from MindConnect MQTT Agent](https://documentation.mindsphere.io/MindSphere/howto/howto-send-data-from-mqtt-agent.html) - [Event Management API Service](../advanced-eventmanagement/api-eventmanagement-overview.html) - [Managing CA Certificates](https://documentation.mindsphere.io/MindSphere/howto/howto-managing-ca-certificates-using-api.html) - [MindConnect MQTT Broker](https://documentation.mindsphere.io/MindSphere/concepts/concept-mindconnect-mqtt-broker.html) # MindConnect Async API – Samples The code given below is just a .json representations of the Timeseries format. ## Topic Structure MQTT Broker offers below described topics: `tc/{tenantId}/{clientId}/o/mc_v3/ts` ## Ingest timeseries data Timeseries Message: Message contains an array of samplings. Each sampling consists of a timestamp and an array of value objects. Each value object consists of data point id, value and quality code. ```json { "timeseries": [ { "timestamp": "2020-08-01T00:15:10.100Z", "values": [ { "dataPointId": "rotationPerMinute", "value": "600", "qualityCode": "0" }, { "dataPointId": "temperature", "value": "1.3", "qualityCode": "-1" }, { "dataPointId": "gearPosition", "value": "1", "qualityCode": "0" }, { "dataPointId": "curve", "value": "381,2776,6586,9362,11268,12139,12247,12574,12846", "qualityCode": "0" } ] } ] } ``` # MindConnect MQTT – API Overview ## Region Europe 1 [Download OpenAPI Specification](/insights-hub/api_specs/mindconnectmqtt-v3-0-2-xcelerator.yaml) # MindConnect MQTT - Usage quota and limits MindConnect MQTT APIs are restricted with following limits. | API Endpoint | Throttling/Restriction Criteria | XS | S | M | L | XL | | ---------------------------------------- | ------------------------------- | --- | --- | --- | --- | --- | | Post caCertificates | Requests in 30 Seconds | 2 | 2 | 2 | 2 | 2 | | Post caCertificates | Requests in 1 Hour | 10 | 10 | 10 | 10 | 10 | | Get caCertificates | Requests in 30 Seconds | 10 | 10 | 10 | 10 | 10 | | Get caCertificates | Requests in 1 Hour | 30 | 30 | 30 | 30 | 30 | | Get caCertificates/{id} | Requests in 30 Seconds | 10 | 10 | 10 | 10 | 10 | | Get caCertificates/{id} | Requests in 1 Hour | 30 | 30 | 30 | 30 | 30 | | Delete caCertificates/{id} | Requests in 30 Seconds | 2 | 2 | 2 | 2 | 2 | | Delete caCertificates/{id} | Requests in 1 Hour | 10 | 10 | 10 | 10 | 10 | | Post caCertificates/{id}/verify | Requests in 30 Seconds | 2 | 2 | 2 | 2 | 2 | | Post caCertificates/{id}/verify | Requests in 1 Hour | 10 | 10 | 10 | 10 | 10 | | Get caCertificates/{id}/registrationCode | Requests in 30 Seconds | 2 | 2 | 2 | 2 | 2 | | Get caCertificates/{id}/registrationCode | Requests in 1 Hour | 10 | 10 | 10 | 10 | 10 | | Post agentCertificates | Requests in 30 Seconds | 10 | 10 | 10 | 10 | 10 | | Get agentCertificates | Requests in 30 Seconds | 30 | 30 | 30 | 30 | 30 | | Get agentCertificates | Requests in 60 Seconds | 40 | 40 | 40 | 40 | 40 | | Get agentCertificates/{id} | Requests in 30 Seconds | 30 | 30 | 30 | 30 | 30 | | Get agentCertificates/{id} | Requests in 1 Hour | 40 | 40 | 40 | 40 | 40 | | Delete agentCertificates/{id} | Requests in 30 Seconds | 5 | 5 | 5 | 5 | 5 | | Delete agentCertificates/{id} | Requests in 1 Minute | 20 | 20 | 20 | 20 | 20 | Note Rate Limits can cause discontinuity in the use case flow. Ensure to check the rate limits before implementing APIs. # MindConnect MQTT API ## Idea The MindConnect MQTT API provides functionality for applications to manage the certificates for securely connecting the MQTT agents with Industrial IoT. With appropriate authentication, the API can be easily integrated into applications hosted in an enterprise system or likewise. For further information about the MindConnect MQTT Sync API, refer to the [MindConnect MQTT API specification](api-mindconnect-mqtt-api.html). Info - The MindConnect MQTT Service is currently available in region Europe 1. - MindConnect MQTT Service is not available for VPC. ## Access For accessing this service, your application or service needs to have the respective roles listed in [MindConnect MQTT roles and scopes](https://documentation.mindsphere.io/MindSphere/concepts/concept-roles-scopes.html#mindconnect-mqtt-api). ## Basics ### CA Certificate All MQTT-based devices that wish to connect to Insights Hub must authenticate using a unique certificate identity. ### Registration Code The user needs to get the registration code to use it as the common name of the verification certificate. ### Verification Certificate For Insights Hub to make sure that the certificate uploader also possesses the corresponding private key, the user needs to prove the possession of the private key by issuing a verification certificate with the common name provided by Insights Hub using the private key. ### Auto-Generated Agent Certificate The MQTT agent needs to authenticate with an agent certificate. MindConnect MQTT agents can request auto-generated agent certificate using the "create auto-generated agent certificate" endpoint. It is available in the region Europe 1. ### Agent Certificate An agent can be onboarded using an agent certificate issued by the environment's CA certificate. To successfully connect, onboard, and communicate with the MQTT broker, each client needs to use a `clientId` in the following format: `=_` ## Features The MindConnect MQTT exposes its API for realizing the following: - Upload a new CA certificate - Get the uploaded CA certificates - Get the CA Certificate with id - Delete the CA certificate by id - Verify the existing CA certificate - Get a CA certificate's registration code The following tasks are currently available in the region Europe 1: - Create auto-generated agent certificate - Get auto-generated agent certificates - Get auto-Generated Agent Certificate with id - Delete auto-Generated Agent Certificate by id ## Limitations In order to give optimal performance, MindConnect MQTT API provides technical limits on API usage and resources which needs to be incorporated while using it. API technical limits are documented in the [MindConnect MQTT - Usage quota and limits](api-mindconnect-mqtt-apiratelimits.html) chapter. The following are the technical limits for resource usage: | Resource | Technical Limit | | --------------------------------------------- | --------------- | | Maximum CA certificates count per environment | 2 | Note These limits are enforced on Capability Package based environments only. ## Example Scenario The ACME company has an IT department regulating the security of the enterprise. The security system manages the issuing of the device certificates using an enterprise application. This system has authorized the administrators to upload the CA and verification certificates using these APIs. This application issues the device certificates to IT users when they want to connect the device. ## Related Links - [Managing CA Certificates using MindConnect MQTT API](https://documentation.mindsphere.io/MindSphere/howto/howto-managing-ca-certificates-using-api.html) - [Managing CA Certificates using UI](https://documentation.mindsphere.io/MindSphere/howto/howto-managing-ca-certificates.html) - [Obtain Auto-Generated Agent Certificate](https://documentation.mindsphere.io/MindSphere/howto/howto-obtaining-auto-generated-agent-certificate.html) - [MindConnect MQTT Broker](https://documentation.mindsphere.io/MindSphere/concepts/concept-mindconnect-mqtt-broker.html) # Remote Service – API Overview ## Region Europe 1 [Download OpenAPI Specification](/insights-hub/api_specs/remote-services-xcelerator-3-0-3.yaml) # Remote Services API ## Idea Remote Services provide APIs that facilitate secure connectivity to devices on the shopfloor for remote technicians and experts by means of Fine Grained Authorization Control (FGAC). These APIs allow modelling isolated organization structures in user-defined tree like structures, where the devices to be accessed form the leaves of the tree. In addition to modelling organization structures, these APIs allow privileged users to provide grants to other users to have access to certain parts of the organization structure. ## Access For accessing this service, you need to have the the following Insights Hub roles: - `mdsp:core:vpnrts.serviceowner` - `mdsp:core:vpnrts.orguser` ### Service Owner Service owner is the role that allows tenant-wide operations such as creating isolated organizations, so that data privacy across different organizations can be achieved. The purpose of the service owner role is to facilitate initiation and supporting of organizations. This role is only given to a user that resides in the organization which is the owner of the tenant. ### Organization User Organization user (orguser) is the role that regular users should have in order to be able to access the Remote Services APIs. This role provides all the necessary access rights for the users to model an organization and perform all other organization level functionalities. #### Remote Services endpoints Remote Services endpoints that belong to users or devices need internet connectivity that allows them to access Remote Services APIs using the HTTPS protocol. ## Basics ### Organizations By default, a tenant is pre-initialized with a single organization called "Service Provider Organization". This is a special organization that every tenant has and all the service owners are automatically onboarded to. In addition, a service owner may decide to introduce additional organizations into their tenant depending on their needs. A typical scenario is that a service provider has a business relationship with another physical organization and they would like to onboard this 3rd party business as an isolated organization. Initially, as the creator of the organization, the service owner will have access to it and may help create the organization structure. New users can be onboarded to the new organization and they may be given access rights to the organization. Typically, the organization is eventually handed over to its owner and the access from Service Provider Organization is only granted via the means of external users. #### Organization Owner Organizations have an internal organization owner role. By default, the service owner who creates the organization is automatically given organization ownership so that they can perform all the necessary operations across the organization. Organization owners can add or remove other organization owners, and sometimes even themselves. #### Service Provider Organization Service Provider Organization is a special organization in a tenant. It cannot be deleted. All the service owners are automatically onboarded to this organization. External users can only be referred from the service provider organization. If an already existing user who was onboarded to a different organization is given the Insights Hub role for service owner, this is considered an erroneous situation since the users can only have a single home organization. For this user, the access to the Service Provider Organization is not given and the associated Insights Hub role is automatically removed. ### External Users External users of an organization are external to the organization. External users may only be originating from the Service Provider Organization. External users are typically experts who provide assistance in maintaining a device on the shopfloor. A newly created organization typically has no users of its own. The only user who will have access to it is the service owner who created it. But, this access is given via the external user status of the service owner as the creator of the organization. This access is given automatically as part of organization creation functionality. External users are referred to an organization by a service owner. Once they are referred, an organization owner in the respective organization may do one of the following: - Admit this referred external user, which enables others in the organization to allow access to this external user. - Alternatively, an organization owner may reject the external user, which causes the referral to be terminated. In this case, the external user will not be available in the organization. Service owner may refer the same user again in future, if necessary. - Another alternative for organization owners is to revoke an admitted external user. All the access granted to this external user will be removed while preserving their role as an external user, so that the same user can be admitted again in the future. Hence, revoking does not require referral by the service owner. An external user with revoked access can be re-admitted. Lastly, a service owner may at any point rescind referral of an external user which results in the removal of the external user status of the user as well as all the access granted in the organization. ### Users In contrast to external users, organizations also have their own users. Every user in the system belongs to a particular organization. Privileged users may add new users to an organization. Upon onboarding to an organization, users are automatically given `orguser` role in Insights Hub, which allows them to access the Remote Services APIs. #### User Roles Users assume different roles, which are determined by the grants and access rights that are given to them by the privileged users. A user may have the following roles: - **Organization Owner:** Organization ownership allows users to perform organization level operations. There are mainly two categories of functionality that requires organization ownership role: external user management and organization management. Organization owner role is provided to users by means of making them organization owners (by other organization owners). - **Organization Admin:** Organization admin role is given via an FGAC mechanism called grants. Once a user is given an organization admin grant, they will have this role in the organization. This role allows users to perform administration tasks within the organization including multiple categories of functionalities such as user management, user group management, grant management, connector management, product, node and site management, etc. With tasks pertaining to organization structure, users with organization admin grant will be limited to those portions of the organization that relevant grant allows them access to. - **Site Owner:** Site owner role is also given via the grants mechanism. A user with a site owner grant will have the site owner role in the organization. Site owners are typically responsible for administrative tasks in a site as well as providing temporary access to devices in the site by remote technicians. - **Remote User:** Remote user role is given via the grants mechanism. Remote users are allowed to establish connections to devices that reside in remote sites. Just like in other cases, the provided grants determine the level of access for the given remote user. #### Account Status Users have an associated account status information stored alongside their data. The account status for a user is typically set to ACTIVE which signifies that the user is able to access the Remote Services functionalities in accordance with the level of access granted to them by privileged users. However, there are additional status values possible: - **SUSPENDED:** A privileged user (organization admin) may suspend a user's account to prevent further access by the respective user. Once a user is suspended, the account status must be explicitly modified by an organization admin to later make the user active again. - **EXPIRED:** Typically, the users will not have accounts that expire. However, it is possible for organization admins to activate users while also setting an expiration date for the future. The respective user will have to be ACTIVE until the expiration date, and then their account status will be set as EXPIRED, which will prevent them from accessing the system. An organization admin may re-activate this user at this point. - **RESTRICTED:** Account status for a user will be set to RESTRICTED upon detection that the user has lost their Insights Hub roles for Remote Services, which would prevent them from using the Remote Services APIs. This situation may occur if a tenant admin removes all necessary roles. In this case, the user can be re-activated by an organization admin or by re-assigning the necessary Insights Hub role to the user. ### User Groups User groups allow simplifying management of user rights for organization admins. Organization admins may introduce new user groups and assign users to these user groups. It is possible to provide grants to user groups which automatically allows users within to acquire these grants. ### Products Each organization has a product hierarchy defined by the organization admin. Products are means to define device types and families in Remote Services. Each product may have its descendants represented by child products. Products are used as part of the grants, to further restrict the given access to a particular product family or device type. Products have an identifier information and each product has its parentID. ### Nodes Each organization has a node hierarchy defined by the organization admin. Nodes are organized in a tree like structure and each node may have multiple children. The way the organization needs to be structured is completely determined by the organization admins. Any logical tree-like structuring is possible by using nodes. A node may contain other nodes or sites underneath. The nodes have an identifier information and each node has its parentID. ### Sites Sites would correspond to the structural element in organizations that are connected to nodes, and they would house the devices. Sites can only have devices in them, hence it is not possible to create other sites or nodes under a given site. In the tree-like organization structure, they are the level immediately above the leaf level (device level). The site is actually a special kind of a node. Sites have an identifier information, which corresponds to their ID as node. Instead of a parentID, sites have the nodeID information which corresponds to the node contains the site. ### Connectors Connectors define the actual connection details. Connectors are defined by organization admins. Once a connector is defined, then it is available for the entire organization. Any organization admin may assign this connector to a device they have access to. Assignment of a connector to a device enables remote users to establish connections using this connector on the device. At its core, a connector definition contains information about the TCP ports that should be used for the tunnels, in addition to some properties of the underlying protocol. Depending on the type of the connector, the required information varies. The available connector types are as below: - **DTT:** This is the most generic connector type that allows tunneling of TCP traffic originating from service endpoint towards a device where the connector is assigned to. - **RDP:** This is a connector type that builds on top of DTT to provide additional convenience to users by providing RDP related properties such as domain name, user name, resolution, etc. - **VNC:** This is a connector type that builds on top of DTT to provide additional convenience to users by providing VNC related properties. - **SSH:** This is a connector type that builds on top of DTT to provide additional convenience to users by providing SSH related properties. - **Proxy Unaware (PU):** This is a connector type that allows clients to connect to remote devices seamlessly even if the clients cannot cope with intermediate proxies. - **DTT-R:** This is similar to DTT. However, the goal is to provide device to device tunneling. DTT-R stands for DTT-Reverse. - **Web Application:** This connector type allows tunneling HTTP based applications to be accessed by remote users. ### User Grants Grants allow the users to have access to parts of the organization while also specifying the kind of access. User grants are the kind of grants that are directly given to individual users. A user grant contains the following: - **userID:** This specifies the user to which this grant is given to. - **role:** The access right given to the user with this grant. Alternative roles that can be given are: - **REMOTE_USER** - **SITE_OWNER** (Site owner grant can only be given to a site) - **ORGANIZATION_ADMIN** - **nodeID:** The node at which the user is given the grant. If there are any children nodes, the user will also have access to those as well. This may either be a node or a site. - **productID:** The product hierarchy level the user is allowed access to. This will specify which kind of devices in the organization structure the user will have access to. ### User Group Grants Similar to user grants, user group grants allow access to parts of the organization, but for user groups. User group grants are inherited by the users assigned to the respective user group. The pieces of information for a user group grant are as below: - **userGroupID:** which specifies the user group to which this grant is given to. - **role:** access right given to the user group with this grant. Alternative roles that can be given are: - **REMOTE_USER** - **SITE_OWNER** (Site owner grant can only be given to a site) - **ORGANIZATION_ADMIN** - **nodeID:** The node at which the user group is given the grant. If there are any children nodes, the user group will also have access to those as well. Note that this may either be a node or a site. - **productID:** The product hierarchy level the user group is allowed access to. This will specify which kind of devices in the organization structure the user group will have access to. ## Features - Organization management - User management - External user management - User groups - Product management - Node management - Site management - Connector management - User grant management - User group grant management - Session management - Parameter management \*\[FGAC\]:Fine Grained Authorization Control is a security approach that provides detailed and specific permissions to users based on their roles, attributes, or contextual factors. Unlike coarse-grained control, which typically offers broad access permissions (e.g., "admin" or "user"), fine-grained control allows for more precise and nuanced access management. # Identity Management – API Overview ## Region Europe 1 [Download OpenAPI Specification](/insights-hub/api_specs/identitymanagement-v3-3-3-xcelerator.yaml) # Identity Management ## Idea The Identity Management Service manages environments, users and groups within Insights Hub. It enables customers to access the UAA service used within Insights Hub for identity management and authorization. ## Access For accessing this service, you need to have the respective roles listed in [Identity Management roles and scopes](https://documentation.mindsphere.io/MindSphere/concepts/concept-roles-scopes.html#identity-management). ## Basics ### Environments An environment is an organization-specific logical environment for your data. An environment typically represents a legal entity, such as a company or corporation. Insights Hub is a multi-tenant architecture. An environment comprises up to two zones for identity management and access control: - **User zone**\ Every environment has a user zone. It enables users of the environment to log in and use applications the company has subscribed to. Administrators of the environments can manage the users and assign roles in the user zone to provide users with access to subscribed applications. - **Provider zone**\ The provider zone represents an additional environment required for web application and API development and operation. Every DevOps plan environment has a provider zone, which stores all information related to access management required for either development or operations (e.g., roles, permissions, applications, technical users). As environment names are global resources, they must be unique across all environments. Each user environment needs to define at least one administrator for managing the users and roles. ### Users and Groups Every environment has its own users and has roles available depending on the applications it is subscribed to. A role represents a grouping of permissions required to access an application. By modeling roles as System for Cross-domain Identity Management (SCIM) groups, the User and Roles management follows the SCIM standard. Currently, within the user zone, SCIM groups are only used to represent roles. In addition, SCIM groups may represent user groups (for managing sets of users), data groups (for managing end customer access to data, assets, etc.) and permissions (for managing more fine-grained access to resources, only within provider zone of a environment). ### Roles and Scopes The following information is relevant for environments with a provider zone only. If you expose an API for your web application, scopes define the application specific permissions. Scope names typically reflect these permissions in a syntax like: `{apiName}.{permission/action}` The following list shows examples for the CRUD-permissions of an IoT service: - `iot.c` (permission to create objects in IoT) - `iot.r` (permission to read objects in IoT) - `iot.u` (permission to update objects in IoT) - `iot.d` (permission to delete objects in IoT) Scopes are mapped to a specific role. A role name has the following syntax: `mdsp:{tenantName}:{application/apiName}.{roleName/action}` Thus, all scopes above could be mapped to a role called `mdsp:core:iot.admin` Application-specific roles and scopes are defined in provider environments and can be managed within the Developer Cockpit application. ### OAuth Client Info The following information is relevant for environments with a provider zone only. An OAuth client (also called technical user) allows your application to acquire a token to access protected resources without the need of human interaction. This is necessary for performing regular background activities (batch activities) within your application, or if your application is not hosted behind Industrial IoT Gateway and therefore does not receive interactive user tokens in request headers. OAuth clients are defined within the provider zone of your environment and comprises a client ID and client secret, which allows to obtain a token using the client credentials grant (RFC 6749). Note The userID for Identity Management API is not the same as the user_id in a decoded token. Instead, a filter with the user's email address needs to be used to get the specific data on a user. OAuth clients for your provider environment can be acquired as described in the [HowTo self-hosted application](https://documentation.mindsphere.io/MindSphere/howto/howto-selfhosted-api-access.html).\ The client secret should be updated before it expires. For more information, refer to [Client Secret Rotation](https://documentation.mindsphere.io/MindSphere/howto/howto-client-secret-rotation.html). ### External Group mapping This feature offers automatic assignment or un-assignment of Insights Hub roles to the users based on external groups defined in IdP. This update happens on every user log-in. This feature is available with API first approach. To configure this feature, follow the below steps: 1. Add external group as additional attribute in IdP config so that external group attribute comes as a claim in IdP ID token. 1. Define the attribute mapping in Insights Hub IdP configuration using [IdP API](api-identitymanagement-api.html). 1. Create mapping between Insights Hub roles/user group and groups defined in IdP using [External Group Mapping API](api-identitymanagement-api.html). 1. After the user logs into Insights Hub, the user automatically gets assigned to Insights Hub role/user group that is mapped with external group in IdP. ## Features The Identity Management Service exposes its API for realizing the following tasks: - List all users of a environment - Create, get, update, delete users of a environment - Get all roles assigned to the own user - List all SCIM groups of the user zone of a environment - Create, get, update, delete SCIM groups of the user zone of environment - List, add, remove members of a SCIM group of the user zone of environment - Create and update mapping for external groups from IdP to Insights Hub roles ## Example Scenario The administrator of a brewery wants to prepare the environment for the new developers of their web application. Use the Identity Management Service to populate the environment with new users and assign them the roles required for development (For example, `mdsp:core:StandardUser, mdsp:core:Developer`). ## Related Links - [Authentication & Authorization](https://documentation.mindsphere.io/MindSphere/concepts/concept-authentication.html) \*\[UAA\]: User Accounts and Authentication \*\[IdP\]: Identity Provider # Message Broker API – API Overview ## Region Europe 1 [Download OpenAPI Specification](/insights-hub/api_specs/messagebroker-v3-0-1-xcelerator.yaml) ## Idea Message Broker is a service that enables asynchronous communication in the platform, that can be used to decouple applications and services from each other. The Message Broker provides the following functionality: - Subscribe to a topic: Provide a web hook integration from core to 3rd party (https Interface) based on the required events. This will reduce the cost and also the number of calls that need to be done to gather information. - Publish to a topic: Publish new message on given postbox topic only. After a successful subscription, the subscriber can receive messages from the given topic over the webhook URL. Webhook URL provides support for HTTP post-operation only. ## Access By default, the Message Broker scopes are part of the 3rd party application token. ## Features The Message Broker exposes its API for realizing the following tasks: - Creating a subscription - Fetching the subscription - Deleting a subscription - Identifying the right topic to publish your message - Sending messages on the topic and providing the message in the content attribute of the payload. ## Example Scenario Any application which needs to consume Message Broker APIs need to acquire token for that application. In addition, it is required to call the Message Broker subscriber API to subscribe to the specific event by providing the 3rd party web hook uri (only post operation is supported). Once the subscription is created, the application will start receiving event notification on the provided web hook uri. ## Related Links - [Token Manager](../exchange-tokenmanager/api-tokenmanager-overview.html) - [Receiving Notifications](https://documentation.mindsphere.io/MindSphere/howto/howto-receive-notifications.html) # Message Broker Service– Samples ## Subscribing Events Note This is applicable only for Early access customers. You can choose to be notified for events of your interest. It is also possible to subscribe for event notifications generated based on event changes. Multiple subscriptions for the same event can be created for each version of the application. ### Subscribe to the event notification **Creating or updating Event Subscription:** Use the following endpoint: ```http PUT /subscribers/{id}/versions/{versionId}/topics/{topicId} ``` ```http Content-Type: application/json ``` After successful subscription, the subscriber can receive messages from the given topic over the webhook uri. Webhook uri should provide support for http post operation only. Request example: ```json { "uri": "https://gateway.eu1.mindsphere.io/api/thirdpartyapp-tenantname/v1/mywebhookURL" } ``` Response example: ```json { "uri": "https://gateway.eu1.mindsphere.io/api/thirdpartyapp-tenantname/v1/mywebhookURL" } ``` ## Getting the details of events subscribed You can get the details of the subscription to which you are already subscribed. Use the following endpoint: ```http GET /subscribers/{id}/versions/{versionId}/topics/{topicId} ``` Response example: ```json { "uri": "https://gateway.eu1.mindsphere.io/api/thirdpartyapp-tenantname/v1/mywebhookURL" } ``` ## Deleting the subscription If you do not want to receive any further notifications, you can delete the created subscription. ```http DELETE /subscribers/{id}/versions/{versionId}/topics/{topicId} ``` Response example: ```http Status 204 No Content ``` ## Publishing message on topic of Interest **Publishing the message:** Publishes new messages to the given topic. Use the following endpoint: ```http POST /topics/{id}/sendmessage ``` ```http Content-Type: application/json ``` Request example: ```json { "content": "string" } ``` Response example: ```http Status 201 No Content ``` # Message Broker Topics | Topic Name | Description | | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `mdsp.core.timeseries.v1.pubsub.data.ingest` | IoT Time Series Subscription Notification for arrival of new data. For more information, refer to [Notification for New Arrival of Timeseries Data](../iot-iottimeseries/api-subscription-notification-overview.html) section. | | `mdsp.core.eventmanagement.v1.pubsub.eventnotification` | IoT Events notification for changes. For more information, refer to [Notification for Event Creation or Update](../advanced-eventmanagement/api-eventmanagement-subscription-notification-overview.html) section. | | `mdsp.core.provisioning.v1.pubsub.provisioningnotification` | Notification for provisioning. | | `mdsp.core.assetmanagement.v1.pubsub.aspecttype.ext.notification` `mdsp.core.assetmanagement.v1.pubsub.assettype.ext.notification` `mdsp.core.assetmanagement.v1.pubsub.asset.ext.notification` | IoT Asset Model change notifications. For more information, refer to [Notification for Asset Model Changes](../advanced-assetmanagement/api-assetmanagement-subscription-notification-overview.html) section. | # OAuth Authorization Server – API Overview ## Region Europe 1 [Download OpenAPI Specification](/insights-hub/api_specs/oauthauthorizationserver-v3-2-xcelerator.yaml) # OAuth Authorization Server (API) The OAuth Authorization Server provides APIs for authentication and authorization. In particular, it provides APIs for requesting access tokens and retrieving the public key used for signing Json Web Tokens issued by the OAuth Authorization Server. # Attribute Based Access Control (ABAC) configuration in SDS through Policy Conditions Note ABAC is supported only in Virtual Private Cloud environments. SDS service provides fine-grained access control using policies, where the Tenant Administrator configures the resources inside policies. These resources can have attributes or properties i.e. metadata key values. Similarly, users can also have their own attributes or properties. More granular and flexible access control of resources can be managed using resource/user attributes or properties. This authorization strategy is known as ABAC. Tenant administrators and application developers can use the SDS policy conditions to further refine the access based on certain attribute conditions. You can realize ABAC configuration in SDS through policy conditions as explained below. ## Policy Conditions Policy conditions allow you to manage access based on resource or user attribute or property values of the resource that is protected. Attribute in policy conditions refers to any property or metadata-key associated with a file or folder. For example, 'BusinessSensitivity', 'CountryOfOrigin' can be considered as attributes or properties or metadata-keys. Policy condition consists of one or more logical expressions which are combined through logical conjunction operators. ### Example ```json { "owner":"tenantA", "id":"8b64e3a0-a315-4eed-babc-58a06cabe614", "name":"Simulation_Files_Policy", "active":true, "description":"Policy to provide access to simulation related files.", "subjects":[ "mdsp:core:identitymanagement:gbl:tenantA:usergroup:mdsp_usergroup:AllSimulationUsers" ], "rules":[ { "name":"Rule1", "actions":[ "mdsp:core:idl:prefix:create", "mdsp:core:idl:prefix:delete", "mdsp:core:idl:prefix:read" ], "resources":[ "mdsp:core:idl:gbl:tenantA:prefix:/data/ten=tenantA/PLISimulationData" ], "propagationDepth":-1, "conditions":[ { "resourceType":"mdsp:core:idl:prefix", "expression":"prefix.Global.countryOfOrigin eq 'IN' AND prefix.SAPData.businessSensitivity eq user.clearanceLevel" } ] } ] } ``` In the above example policy, a new JSON element "conditions" has been added into the policy schema, which contains 2 sub-elements "resourceType" and "expression". Condition is an expression(s) with selection criteria defined on resource attributes or user attributes, based on which access to the resource(s) is determined. For example, in IDL, metadata keys are considered as resource attributes. - `ResourceType` represents the type of resource for which expression has been defined. Resource attributes which belong to the given `resourceType` can only be used in the expression. For example, if value of `resourceType` is `mdsp:core:idl:prefix` then, all the metadata keys or resource attributes in underlying expression should have following format: `prefix..`. For example: `prefix.SAPData.businessSensitivity`. - `Expression` is a combination of resource attributes and user attributes compared against some value. Larger expressions can be formed by joining individual expressions using logical operators (AND, OR, NOT). Details of valid resource attributes/user attributes and supported operators can be found be in the owning service. For example, metadata keys/resource attributes and supported operators around resourceType `mdsp:core:idl:prefix` are defined in IDL API documentation. Similarly, user attributes definitions and supported operators are defined in "Identity and Access Management" (IAM) API documentation. For more details about these API specifications, please refer to the relevant documentation. ### Constraints - Currently, only one expression is allowed per `resourceType` - `resourceType` cannot be duplicated inside conditions - Currently, only supported `resourceType` is IDL prefix (`mdsp:core:idl:prefix`). Asset resourceType is not yet supported - A maximum of 15000 characters are allowed in a condition `expression` - Policy conditions should only be used when SDS basic policy configuration is no longer sufficient, because it performs a more elaborate policy search and evaluation that demands more processing power and time - Any changes done in IDL metadata/User Attribute definitions or assignments, it may take up to 1 hour to reflect in Policy configuration due to caching mechanism ### Supported Operators in Policy Conditions | Type of operator | Operator | Symbols | | ---------------- | ------------ | ------------- | | Logical Operator | OR | 'or' or ' | | Logical Operator | AND | 'and' or '&&' | | Logical Operator | NOT | 'not' or '!' | | Binary Operator | Equal to | 'eq' or '==' | | Binary Operator | Not equal to | 'ne' or '!=' | | In Operator | IN | in | | Not In Operator | NOT IN | not in | | Other | Parenthesis | '(' and ')' | ### Supported Data Types in Policy Conditions | Data Type | Symbols | | ----------- | ------------------------------------------------ | | Enum | 'SOME_ENUM_VALUE' | | Enum List | ('SOME_ENUM_VALUE', 'SOME_OTHER_ENUM_VALUE') | | String | 'some_string_value' | | String List | ('some_string_value', 'some_other_string_value') | Exceptions There are some exceptions to supported data types in policy conditions: - "In" and "Not in" Operators are not supported for the following combinations. - List `"in"/"not in"` List - List `"in"/"not in"` Enum/String - String/Enum `"in"/"not in"` String/Enum ### Error Handling Scenarios in Policy Conditions 1. **Sample Invalid Expression**: `prefix. global.country : eq 'GB'` Here, ':' is unsupported character in the expression **Error response**: ```Json { "errors": [ { "code": "mdsp.core.resourceaccessmanagement.validation.malformedExpression", "message": "Malformed Expression provided. Please correct the expression syntax and try again.", "messageParameters": [ { "name": "offendingSymbol", "value": ":" }, { "name": "expression", "value": "prefix.global.country : eq 'GB" }, { "name": "resourceType", "value": "mdsp:core:idl:prefix" } ], "logRef": "6478534b8ef9a8ef16a7eda2ce3e7231" } ] } ``` In case unsupported characters are found in expression. 1. **Sample Invalid Expression**: `prefix.global.country eq !'GB'` Here, Not(!) Operator is incorrectly placed in the expression. **Error response**: ```Json { "errors": [ { "code": "mdsp.core.resourceaccessmanagement.validation.malformedExpression", "message": "Malformed Expression provided. Please correct the expression syntax and try again.", "messageParameters": [ { "name": "resourceType", "value": "mdsp:core:idl:prefix" }, { "name": "expression", "value": "prefix.global.country eq !'GB" }, { "name": "offendingSymbol", "value": "!" } ], "logRef": "647854481fcbb128ef84d7798151c1f5" } ] } ``` The system considers this expression as invalid at the beginning and therefore so sends (!) as an offending symbol. 1. **Sample Invalid Expression**: `(prefix.global.country eq 'GB' and prefix.global.country eq 'GB'` `prefix.global.country eq 'GB' and (prefix.global.country eq 'GB'` If closing parenthesis is missing `prefix.global.country eq 'GB'` and after part of valid expression if any known keyword is used by mistake. **Error response**: ```Json { "errors": [ { "code": "mdsp.core.resourceaccessmanagement.validation.malformedExpression", "message": "Malformed Expression provided. Please correct the expression syntax and try again.", "messageParameters": [ { "name": "resourceType", "value": "mdsp:core:idl:prefix" }, { "name": "offendingSymbol", "value": "" }, { "name": "expression", "value": "prefix.global.country eq 'GB' and (prefix.global.country eq 'GB'" } ], "logRef": "647991aaf9e2343b6dabf18ca31a2608" } ] } ``` Missing ) parenthesis shows offending symbol as EOF by the system which can be manually updated. After part of valid expression if any known keyword is used mistakenly, system gives EOF as offending symbol. 1. **Sample Invalid Expression**: `pref.global.country eq 'GB'` `user11.country eq 'GB'` Invalid start of compound Expression. **Error response**: ```Json { "errors": [ { "code": "mdsp.core.resourceaccessmanagement.validation.invalidExpression", "message": "metaDataKey/userAttribute in the given expression pref.global.country should start with prefix/user respectively.", "messageParameters": [ { "name": "resourceType", "value": "mdsp:core:idl:prefix" }, { "name": "expression", "value": "pref.global.country eq 'GB'" } ], "logRef": "647855e952352a455f14cea5e581ae05" } ] } ``` Compound Expression can either start with prefix or user. 1. **Sample Invalid Expression**: `user.xxxx eq 'IN'` Invalid User Attribute. **Error response**: ```Json { "errors": [ { "code": "mdsp.core.resourceaccessmanagement.validation.invalidUserAttribute", "message": "Invalid user attribute xxxxx found in part of given expression user.xxxxx", "messageParameters": [ { "name": "userAttribute", "value": "xxxxx" }, { "name": "expression", "value": "user.xxxxx > 'IN'" }, { "name": "resourceType", "value": "mdsp:core:idl:prefix" } ], "logRef": "64788932907235062517cb154df5c348" } ] } ``` 1. **Sample Invalid Expression**: `prefix.global.citylist in user.city` Invalid Condition as List in String/Enum combination is not supported **Error response**: ```Json { "errors": [ { "code": "mdsp.core.resourceaccessmanagement.validation.leftOperandDatatypeNotSupported", "message": "Attribute datatype of left operand not supported for the given operator 'in' in the given expression prefix.global.citylist in user.city", "messageParameters": [ { "name": "expression", "value": "prefix.global.citylist in user.city" }, { "name": "resourceType", "value": "mdsp:core:idl:prefix" } ], "logRef": "64788932907235062517cb154df5c348" } ] } ``` \*\[SDS\]: Secure Data Sharing \*\[ABAC\]: Attribute Based Access Control \*\[IDL\]: Integrated Data Lake # Action Overview and explanation Actions are the functional permissions between an acting [subject] and an acted on [object]. They allow you to fine-tune access patterns to your requirements. ## List of all available actions | Resource Type | Action | Description | | ------------- | ---------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | | mdsp:core:**idl:prefix**:read | Read a prefix (an object in an Integrated Data Lake), to download data from Data Lake | | Prefix | mdsp:core:**idl:prefix**:create | Create a prefix, to upload data into Data Lake | | | mdsp:core:**idl:prefix**:delete | Delete a prefix, to delete data (single object or in bulk) from Data Lake | | Asset | mdsp:core:**assetmanagement:asset**:read | Allows a user to read given asset | | Asset | mdsp:core:**assetmanagement:asset**:write | Allows a user to make modifications on given asset (on which policy is being created). This means - user can update/delete given asset - user can create a new asset with given asset as its parent - user can update its location - user can move given asset under other asset on which user has write access | | Asset | mdsp:core:**eventmanagement:event**:allow | Allows a user to read, create, bulk create and update access to events of corresponding asset | | Asset | mdsp:core:**iotservices:timeseries**:write_normal | Allows a user to push timeseries data for single Asset/Aspect | | Asset | mdsp:core:**iotservices:timeseries**:write_multiassetmultiaspect | Allows a user to push timeseries data for multiple Asset/Aspects | | Asset | mdsp:core:**iotservices:timeseries**:write_bulk | Allows a user to import high frequency timeseries data | | Asset | mdsp:core:**iotservices:timeseries**:write_merge | Allows a user to push timeseries data as patch/merge operation for single Asset/Aspect | | Asset | mdsp:core:**iotservices:timeseries**:read | Allows a user to read ingested or aggregated timeseries data, also used to import timeseries data into Data Lake | | Asset | mdsp:core:**iotservices:timeseries**:delete | Allows a user to delete timeseries data | | Asset | mdsp:core:**iotservices:timeseries**:subscribe | Allows a user to read, create, delete timeseries subscription data | | Asset | mdsp:core:**iotservices:files**:write | Allows a user to perform File write operation (single or multi-part write) | | Asset | mdsp:core:**iotservices:files**:read | Allows a user to perform File read operation (single or multi-part read) | | Asset | mdsp:core:**iotservices:files**:delete | Allows a user to perform File delete operation | ## Dependencies among Actions Some actions have others as a prerequisite. To avoid inconsistencies, they are enforced during creation of policies. Info The Policy Editor will show you these dependencies during configuration. | Action | Depends On | | ---------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------- | | mdsp:core:**assetmanagement:asset**:write | mdsp:core:**assetmanagement:asset**:read | | mdsp:core:**eventmanagement:event**:allow | mdsp:core:**assetmanagement:asset**:read | | mdsp:core:**iotservices:timeseries**:write_normal | mdsp:core:**assetmanagement:asset**:read | | mdsp:core:**iotservices:timeseries**:write_multiassetmultiaspect | mdsp:core:**assetmanagement:asset**:read | | mdsp:core:**iotservices:timeseries**:write_bulk | mdsp:core:**assetmanagement:asset**:read mdsp:core:**iotservices:files**:write mdsp:core:**iotservices:files**:read | | mdsp:core:**iotservices:timeseries**:write_merge | mdsp:core:**assetmanagement:asset**:read | | mdsp:core:**iotservices:timeseries**:read | mdsp:core:**assetmanagement:asset**:read | | mdsp:core:**iotservices:timeseries**:delete | mdsp:core:**assetmanagement:asset**:read | | mdsp:core:**iotservices:timeseries**:subscribe | mdsp:core:**assetmanagement:asset**:read mdsp:core:**iotservices:timeseries**:read | | mdsp:core:**iotservices:files**:write | mdsp:core:**assetmanagement:asset**:read | | mdsp:core:**iotservices:files**:read | mdsp:core:**assetmanagement:asset**:read | | mdsp:core:**iotservices:files**:delete | mdsp:core:**assetmanagement:asset**:read | # Resource Access Management – API Overview ## Region Europe 1 [Download OpenAPI Specification](/insights-hub/api_specs/resourceaccessmanagement-v3-4-0-xcelerator.yaml) # Resource Access Management ## Idea Secure Data Sharing (SDS) enables you to create and manage fine-grained access rights. It is based on industry standard paradigm PBAC, which is an advanced framework to centrally manage permissions on business resources. SDS complements the already existing access management capabilities based on RBAC framework. For Virtual private cloud subscribers, SDS complements the already existing access management capabilities based on ABAC framework. SDS encompasses several services to solve the fine-grained access needs of enterprises. One of the key backend services is RAM, which provides an interface to configure and manage policies for several business objects. ## Access During the first stage of general availability, by default, SDS policies are not activated in your environment. To get access, contact our support team. User Permissions Be aware that through enablement of SDS policies on your environment, all Standard Users and Subtenant Users lose access to all SDS protected resources. You need to create policies to grant access again. For accessing "Policy" in the "Settings" application, you need to have the respective roles listed in [Resource Access Management roles and scopes](https://documentation.mindsphere.io/MindSphere/concepts/concept-roles-scopes.html#resource-access-management-service). By default, these are enabled for users with the role `Tenant-Administrator`. ## Basics Once the Resource Access Management switch is turned ON in the "Settings" application, by default, all access to the resources (Assets, Time Series Data, Events, Files and Folders in the Integrated Data Lake) participating in Resource Access Management is denied. Access can then only be granted via policies. Note Once Resource Access Management is provisioned for your environment, it takes up to 1 hour to reflect the change. Similarly, requests for policy quota upgrades also take up to an hour to reflect. Exceptions There are some exceptions to this rule: - A user having the role `Tenant-Administrator` is allowed unrestricted access to all the assets, events, and timeseries data of that environment for out-of-the-box applications (like Asset Manager, Insights Hub Monitor, etc.). - A user having the role `Data Lake Manager` is allowed unrestricted access to all folders and files in IDL inside this environment. - Third-party or custom applications would still need a valid policy to access resources, irrespective of the `Tenant-Administrator` or `Data Lake Manager` user role. - A technical user (without Interactive User Impersonation) has unrestricted access to all the resources of the environment, irrespective of whether the user is added to a policy. ## Policy A policy, at high level, consists of a mapping of three entities namely, subjects, actions, and resources/resource groups. This specifies who gets access, which resources are accessible, and what actions (like Add, Delete, Read, Write) are permitted. A policy outlines which subjects can perform specific actions on designated resources. Further details about the policy resource groups is mentioned in below section. Set of actions, and resources are bound by a rule, and a policy can have multiple rules. Since a policy always grants access and has no provision to deny access, overlapping policies where the same user may have the access granted via multiple policies, have no impact. A policy can be deactivated any time, and policy is bound to the environment. For more information on Policy schema, refer [Resource Access Management API Specification](api-resourceaccessmanagement-api.html). ## Resource and Resource Group A policy resource represents either an individual business object (Asset or IDL File/Folder) or a Resource Group containing one or more individual business objects. In a nutshell, Resource Group is a container of heterogeneous resources. One resource can be associated with multiple Resource Groups. A Resource Group cannot contain another Resource Group. That means nesting of Resource Groups is not supported. With the following advantages, Resource Groups allow for more flexibility in managing access to member resources: - Within the same policy, administrators can manage a greater number of resources when properly organized. - Policy configuration does not need to be adjusted if only resources are being added or removed from the Resource Group. For more information on how the resource and resource groups are structured in a policy, refer to [Resource Access Management API Specification](api-resourceaccessmanagement-api.html). ## Attribute Based Access Control (ABAC) Note ABAC is supported only in Virtual Private Cloud environments. Secure Data Sharing service provides Fine-Grained Access Control using Policies, where the Tenant Administrator configures the resources inside policies. These resources can have attributes or properties i.e. metadata key values. Similarly, users can also have their own attributes or properties. More granular and flexible access control of resources can be managed using resource or user attributes or properties. This authorization strategy is known as Attribute Based Access Control (ABAC). ABAC takes numerous factors into consideration while allowing access and offers an additional layer of protection over and above basic SDS. It also permits multiple access scenarios with minimum administrative supervision. ABAC is helpful in environments that are growing rapidly and helps with situations where policy management becomes difficult to handle. ## Limits There are certain limits on the amount of policies and few internal objects within a policy. These are listed in [Policy Limits](limits.html). ## Known Issues For any known issues, refer [Known Issues](known-issues.html). \*\[PBAC\]: Policy Based Access Control \*\[RBAC\]: Role Based Access Control \*\[RAM\]: Resource Access Management \*\[IDL\]: Integrated Data Lake # Required Policy actions by Industrial IoT services Protecting the business resources (like Assets, Events, IoT Files, TimeSeries data, Data Lake files/folders) requires certain mandatory actions to be specified in Policy definitions, failing which, the operation will be denied. API access is controlled at different levels, role-based and policy-based. ## Role based access control To access any API, the user needs to have the respective roles listed in [Roles & Scopes for Applications](https://documentation.mindsphere.io/MindSphere/concepts/concept-roles-scopes.html). ## Policy based access control Secure Data Sharing (SDS) enables you to create and manage fine-grained access rights. It is based on industry standard paradigm Policy Based Access Control (PBAC). Here, policy describes a given set of subjects and is allowed to perform a given set of actions on a specified set of resources.\ If tenant is SDS enabled, then due to "denial by default" approach, even if user has the required roles to manage resources, the user would not be able to perform operations on some of the service APIs. More information about SDS and how to create policy can be found in [Policies.](api-resourceaccessmanagement-overview.html) Note - For more details about actions and their dependencies, refer [Action Details](api-resourceaccessmanagement-actions-list.html#list-of-all-available-actions) and [Dependencies among Actions](api-resourceaccessmanagement-actions-list.html#dependencies-among-actions). - Dependent action(s) need to be explicitly added in Policy definition along with the parent action. The following sections describe the mandatory actions required by respective service APIs. ## Asset Management Here is the list of Asset Management APIs that are protected through Resource Access Management policies, along with the required fine-grained actions. For further API details, refer [Asset Management API Specification](../advanced-assetmanagement/api-assetmanagement-api.html). | API | Action Required | | ----------------------------------------- | ---------------------------------------------------------------------------------------------- | | GET /assets | mdsp:core:assetmanagement:asset:read | | POST /assets | mdsp:core:assetmanagement:asset:write (on parent asset) | | GET /assets/{id} | mdsp:core:assetmanagement:asset:read | | PUT /assets/{id} | mdsp:core:assetmanagement:asset:write | | PATCH /assets/{id} | mdsp:core:assetmanagement:asset:write | | DELETE /assets/{id} | mdsp:core:assetmanagement:asset:write | | POST /assets/{id}/move | mdsp:core:assetmanagement:asset:write (on new parent asset; and also on the asset being moved) | | PUT /assets/{id}/fileAssignments/{key} | mdsp:core:assetmanagement:asset:read | | DELETE /assets/{id}/fileAssignments/{key} | mdsp:core:assetmanagement:asset:read | | GET /assets/{id}/variables | mdsp:core:assetmanagement:asset:read | | GET /assets/{id}/aspects | mdsp:core:assetmanagement:asset:read | | PUT /assets/{id}/location | mdsp:core:assetmanagement:asset:write | | DELETE /assets/{id}/location | mdsp:core:assetmanagement:asset:read | Exceptions - Only /assets APIs are SDS enabled, /assettypes and /aspecttypes APIs are not. - The Root Assets are the assets with the asset type as `core.basicenterprise`, these assets will be visible without any policy definition. - A user having the role `Tenant-Administrator` is allowed unrestricted access to all the assets of the tenant. - A technical user has unrestricted access to all the assets of the tenant. ## Event Management Here is the list of Event Management APIs that are protected through Resource Access Management policies, along with the required fine-grained actions. For further API details, refer [Event Management API Specification](../advanced-eventmanagement/api-eventmanagement-api.html). | API | Action Required | | ---------------------- | ------------------------------------- | | GET /events | mdsp:core:eventmanagement:event:allow | | POST /events | mdsp:core:eventmanagement:event:allow | | GET /events/{eventId} | mdsp:core:eventmanagement:event:allow | | PUT /events/{eventId} | mdsp:core:eventmanagement:event:allow | | POST /createEventsJobs | mdsp:core:eventmanagement:event:allow | Exceptions - Only events APIs are SDS enabled. `EventType` APIs are not SDS enabled. - Assets with the asset type such as `core.basicenterprise`, `core.basicsubtenant`, `core.sharerenterprise` are root assets. - All events operations for such root assets can be performed without any policy definition. - A user having the role `Tenant-Administrator` is allowed unrestricted access to all the events of the tenant. - A technical user has unrestricted access to all the events of the tenant. ## Integrated Data Lake Service Here is the list of Integrated Data Lake Service APIs that are protected through Resource Access Management policies, along with the required fine-grained actions. For further API details, refer [Integrated Data Lake Service API Specification](../iot-integrated-data-lake/api-integrated-data-lake-api.html). | API | Action Required | | -------------------------------- | ------------------------------------- | | POST /generateUploadObjectUrls | mdsp:core:idl:prefix:write | | POST /generateDownloadObjectUrls | mdsp:core:idl:prefix:read | | DELETE /objects/{path} | mdsp:core:idl:prefix:delete | | DELETE /deleteObjectsJobs | mdsp:core:idl:prefix:delete | | POST /timeSeriesImportJobs | mdsp:core:iotservices:timeseries:read | ## IoT File Service Here is the list of IoT File Service APIs that are protected through Resource Access Management policies, along with the required fine-grained actions. For further API details, refer [IoT File Service API Specification](../iot-iotfile/api-iotfile-api.html). | API | Action Required | | ----------------------------------- | ---------------------------------- | | PUT /files/{entityId}/{filepath} | mdsp:core:iotservices:files:write | | GET /files/{entityId}/{filepath} | mdsp:core:iotservices:files:read | | DELETE /files/{entityId}/{filepath} | mdsp:core:iotservices:files:delete | ## IoT Time Series Service Here is the list of IoT Time Series APIs that are protected through Resource Access Management policies, along with the required fine-grained actions. For further API details, refer [IoT Time Series Service API Specification](../iot-iottimeseries/api-iottimeseries-api.html). | API | API method | Action Required | | -------------------------------------------------- | ---------- | ------------------------------------------------------------ | | To ingest timeseries data on single Asset/Aspect | PUT | mdsp:core:iotservices:timeseries:write_normal | | To ingest timeseries data on multiple Asset/Aspect | PUT | mdsp:core:iotservices:timeseries:write_multiassetmultiaspect | | To import high frequency timeseries data | POST | mdsp:core:iotservices:timeseries:write_bulk | | To read ingested timeseries data | GET | mdsp:core:iotservices:timeseries:read | | To delete timeseries data | DELETE | mdsp:core:iotservices:timeseries:delete | | To read aggregated timeseries data | GET | mdsp:core:iotservices:timeseries:read | # Troubleshoot errors with Resource Access Management ## Error scenarios All the possible error details like Http Status code, error code and messages are specified in [Resource Access Management API Specification](api-resourceaccessmanagement-api.html). ## General issues ### Scenario: Policy activated, but not taking effect #### Issue Policy is activated, but somehow not taking the desired effect, i.e. APIs are not honoring the fine-grained access check on certain resources. #### Cause Policy enforcement for the given environment is not enabled. #### Resolution To enable or verify the Policy enforcement on a given environment, contact support team. This enforcement may take up to 1 hour to reflect the change. ### Scenario: Policy for resource subsections #### Issue For resource hierarchy scenarios, a user expects that a subsection of resources are given access through a single resource specified in the Policy. #### Cause The parameter value for Propagation Depth may not be set correctly. #### Resolution Set the Propagation Depth value as **-1** so that the given resource and all its descendants are included in the eligible resources. # Understanding Policy effects Once the policies are in force, access restrictions are applied to the resources (Assets, Files, Events, Timeseries data) through various services and applications. ## Prerequisites - SDS should be provisioned for the environment. - Toggle switch for "Resource Access Management" in the "Settings" application should be turned ON. ## Policy effects By activating the policies, application and service users (apart from Tenant Administrators, Data Lake Managers and Techusers - without Interactive User Impersonation) will be allowed controlled access as defined in the Policy, while accessing the resources as mentioned above. The nature of access restrictions depends on which fine-grained actions are specified in policy definitions. The list of APIs from various services that support fine-grained access management are listed in [Services & Required Actions](api-resourceaccessmanagement-services-actions-map.html). Note After creating/updating an "active" policy, it takes up to 5 minutes to reflect this change for the access checks in Insights Hub platform services. Similarly, any changes in Usergroup operations (like move, rename, add/remove users, etc.) also takes up to 5 minutes to take effect. Note When creating/updating a policy, the validations applied to policy fields will vary based on the policy's active or inactive status. If a policy is marked as active, comprehensive validations will be performed, whereas if a policy is marked as inactive, most validations will be skipped/bypassed. \*\[SDS\]: Secure Data Sharing # Policy definition structure Policy definition consists of meta data information and other key constructs like policy author(s) and target resources along with the fine-grained actions available on those resources. These important constructs are described below as Subject and Rules. ## Subject These are the actors who are allowed to perform certain actions on certain resources. These actors include individual users (email address), user groups. ### User `mdsp:core:identitymanagement:< region >:< tenant >:user:user@example.com` ### User Group `mdsp:core:identitymanagement:< region >:< tenant >:usergroup:administrators` - region\ Region where tenant is hosted, for example eu1, gbl etc. - tenant\ Name of the tenant ## Rules A rule is a container that holds tuple of *actions* and *resources*. A policy can contain multiple rules, and maximum number of rules per policy is mentioned under limits. ### Actions These are the actions, that subjects are allowed to perform on given set of resources. The general syntax for action is: `mdsp:core:< service >:< object >:< action >` Service Name of the service that has exposed this action, and can perform that action on given object. Object Object is the entity, that associated service manages, and exposes certain actions for that object. Object can be same as resource or it can be different from resource. For example, in case of IDL, object and resource are same, i.e. the prefix. The same is applicable for Asset Management service as well. But, for Time Series service, Time Series is the object, but the associated resource for which this action is exposed is an asset. So, in other words, it is an action to allow specific operation on Time Series data of a given asset(s). Action This is a granular action that can be performed on the object, in the context of associated resource. Complete list of such actions are mentioned [Action Overview and explanation](api-resourceaccessmanagement-actions-list.html#list-of-all-available-actions). An action may depend upon another action. For example, a user can perform any operation on any event of an asset, provided user also has read access for that asset. If such is the case, then both of these actions must be present in same policy. If read access on an asset and allow access for event are provided in different policies, then dependency would not be considered as fulfilled and policy creation update would fail. Hence, it is important to consider action dependency during policy creation or update. A dependency matrix is mentioned [here](api-resourceaccessmanagement-actions-list.html#dependencies-among-actions). ### Resources - These are the resources on which a subject specified in the policy is permitted to perform the actions described in the corresponding rule. - The action may not directly be on the mentioned resource, rather the object mentioned in the action, but in the context of given resource, as explained above. The general syntax for a resource is: `mdsp:core:< service >:< region >:< tenant >:< resource >:< resource path >` Currently, asset and IDL prefix are the only resource types supported. Both of these resource types support hierarchy. And, such hierarchy can be specified in a policy as well. Hierarchy can be specified using a special parameter called **propagationDepth**, which is set at rule level, and gets applied to all the resources in that rule. Note For Integrated Data Lake resources, resource path should start with '/' and should not end with '/'. ### Propagation Depth Propagation depth defines how deep into the hierarchy of resources the defined actions given in the same rule are applicable. This parameter is available only at the rule level, meaning the depth specific in this parameter is applicable for all the resources in the given rule. It supports three values: - **All children or -1**: the resource and all its descendants are included in the eligible resources. Use this if you want to grant access to defined subsections of your environment dynamically. - **Only selected or 0**: only the resource mentioned is included in the effective list of resources. Its descendants are excluded. This is used to restrict access to a specific selection of resources only. - **Direct children or 1**: the resource and its direct descendants (only) are included in the effective list of resources. To illustrate it further, consider a plant building, its floors, rooms, and machines in those rooms are modelled in asset hierarchy. | Resource | propagationDepth | Effective Resources | | --------- | ---------------- | ----------------------------------------------------------------------------------------------- | | 2nd-floor | `-1` | - 2nd-floor asset - All room assets on 2nd floor - All machine assets in all rooms on 2nd floor | | 2nd-floor | `0` | 2nd-floor asset ONLY | | 2nd-floor | `1` | - 2nd-floor asset - All room assets on 2nd floor | Best Practice For the scenarios where whole or part of the resources tree structure needs to be given same access privileges (for example: asset hierarchy), then an administrator could specify **Propagation Depth** value as **-1**, thereby avoiding a long list of individual resources. \*\[IDL\]: Integrated Data Lake # Known Issues with Resource Access Management On this page, you will find the most recent aggregation of all known issues related to Resource Access Management (RAM). ## General - If RAM has been activated on your tenant, policies to your users have to be configured.\ Otherwise a user for example with `StandardUser`-role will only see the root assets in his tenant. ### Applications using App Credentials or other technical accounts - If an application uses App Credentials or other technical accounts to interact with the API of Secure Data Sharing enabled services data may be exposed as policies are not enforced for technical accounts as of now. ### Insights Hub Monitor - If the configured policy has gaps in the asset tree (assets missing between), the hierarchy filter is empty.\ In this case, close the hierarchy filter and work with the flat list. - Rules are always displayed, also when the monitored asset is not included in the policy. - The asset info plugin shows the whole hierarchy breadcrumb, even when some assets are not accessible.\ Nevertheless, it is not possible to open such hidden assets or query data for them. - Simple KPIs are always displayed, also when the used asset(s) are not included in the policy. ### Web Components - There are currently no known issues ### Visual Flow Creator - Not supported with active RAM. A user with a policy might not see resources (like assets). However, flows that have been configured to run with these resources will still be able to run. ### Rules - Not supported with active RAM. Rules will be displayed and evaluated even if dependent resources (assets) are not accessible for the user. ### Insights Hub Business Intelligence - With active RAM, the users are not allowed to select a resource if not granted access to it by a policy. However, if they have access to a data source in Insights Hub Business Intelligence, which refers to that resource, they can still view the data included. Review the permissions in Insights Hub Business Intelligence with every policy update to avoid data still being exposed to unauthorized users. ### Predictive Learning - User will view all the assets from asset management and he will be able to create the IoT import jobs on those assets and will be able to use the IoT data datasets irrespective of access control. - User will be able to read IoT data for all assets irrespective of access control and will be able to use it for model development and model execution. - Data Lake data source can be created on any valid data lake resource path irrespective of access control and will be used for model development and model execution. - Jupyter/Zeppelin/Tensorflow workbenches support running the scripts which may call all Insights Hub public API's without any access control restrictions. \*\[RAM\]: Resource Access Management # Policy Limits There are several limits on business entities that should be considered when designing your policies. As the policies interact with several resources like Asset Model, Time Series Data, Events and Integrated Data Lake, these limits ensure that the performance of Insights Hub remains high. There are few limits that applies specifically to the subscribers of Capability Packages. The following table depicts the technical limits on various parameters for all Resource Packs (XS, S, M, L, XL). | Parameter | | XS | S | M | L | XL | | --------------------------- | --- | --- | --- | --- | --- | --- | | No. of Policies per Account | | 50 | 100 | 150 | 200 | 250 | ## Boundary conditions Once the Resource Access Management feature is switched ON through Settings UI application, certain limits are applicable on the amount of various object types, as mentioned below. | Object Type | Limit | Remarks | | ---------------------------------------------- | ----- | ------------------------------------------------------------------------------------ | | Subjects (Users / User Groups) per Policy (\*) | 10 | This represents the aggregated sum of Users and User Groups in a policy | | Rules per Policy | 5 | This represents the number of rules in a policy | | Actions per Rule | 20 | This represents the number of actions in a Rule | | Resources per Resource Group (\*) | 20 | This represents the number of Resources in a Resource Group | | (Resources / Resource Groups) per Rule | 20 | This represents the aggregated sum of Resources and Resource Groups | | Resource Groups per Rule (\*) | 2 | This limit is set within the above aggregated block | | Resource Groups per Account (\*) | 10 | This represents the number of Resource Groups allocated per account | | Resource Group associations across Policies | 5 | This represents the number of Policies to which a single Resource Group can be added | | User Group associations across Policies (\*) | 5 | This represents the number of Policies a User Group can be part of | | Policies per Account (\*) | 50 | This represents the number of Policies allocated per account | (\*) For use cases which do not fit within these limits, kindly contact us via Support Team. Once the allocated quota is exhausted, further creation requests will result in the error `400: Bad request` with an appropriate error message. ## Technical Limits Apart from the above specified boundary conditions, the following technical limits are implemented to ensure reliable system performance. | Object Type | Technical Limit | | ----------------------------------------- | --------------- | | Subjects (Users / User Groups) per Policy | 25 | | Resources per Resource Group | 100 | | Resource Groups per Rule | 4 | | Resource Groups per Account | 50 | | User Group associations across Policies | 10 | | Policies per Account | 350 | On reaching the system limit, further creation requests will result in the error `400: Bad request` with an appropriate error message. ## Policy Modelling Recommendations - On exhausting Policy quota, if there are too many inactive policies, those can be reused or cleaned up for configuring more policies. - Resource Groups are powerful elements for organizing the relevant resources and get better control of the collection. These should be used wherever possible. It also allows managing fine-grained access for a greater number of resources. - For better performance, a given User or User Group should not be part of more than 5 policies. - Similarly, for better performance, a given Resource Group should not be part of more than 5 policies. # Quickstart: Create a policy through Policy Manager (Settings application) ## Prerequisites For creating a policy, ensure that you have `tenantadmin` role assigned. Note During the first stage of general availability, by default, Secure Data Sharing Policies are not activated in your environment. To gain access, please contact support team. Once the support team processes the activation request, it takes around 1 hour to reflect the change (i.e. Policy Editor to appear in "Settings" application). ## Create a Policy and assign users To create a policy, proceed with the following steps: 1. In the "Settings" application, click "Policies" tab in the left navigation. 1. Click "Create Policy". 1. Enter the policy name and description and click "Create policy". 1. Click "Save". 1. To activate the policy, click "Activate". ## Assign users or user groups to the policy To assign users or user group to the created policy, proceed with the following steps: 1. In the "Policies" screen, select the policy for which you want to assign users or user groups. 1. To add users, click "Add users" and to add user groups, click "Add user groups". 1. In the "Edit assignment" window, select the users/user groups to be assigned to this policy and click "Next". 1. Click "Save". 1. To activate the policy, click "Activate". The selected users/ user groups are successfully added to the policy. ## Add rules to the policy To add rules to the policy, proceed with the following steps: 1. In the "Policies" screen, select the policy for which you want to add rule. 1. Click "Add rule". 1. In the "Resource selection" step, select the assets and the corresponding resources and click "Next". 1. In the "Rule Details" step, select the "Propagation Depth" and click "Next". The available options are: "Only selected", "Direct children" and "All children". 1. In the "Actions" step, select the required actions that needs to be enabled for the assigned users/user groups. 1. Click "Save". 1. To activate the policy, click "Activate". After activating the created policy, the user/users assigned to this policy will now be able to view the assigned assets in their tenant. Note After creating/updating an "active" policy, it takes around 5 minutes for this change to take effect for the access checks in platform services. Similarly, any changes in Usergroup operations (like move, rename, add/remove users, etc.) also takes around 5 minutes to take effect. It is possible to edit, delete or deactivate the policy at any time. It is possible to edit or delete the policy by clicking "Edit" and "Delete" button respectively. You can also deactivate the policy at any required time by clicking "Deactivate" button. For more information on policy, refer [Settings documentation](https://documentation.mindsphere.io/resources/html/settings/en-US/index.html). # Quickstart: Create a policy through REST API ## Pre-requisites For creating/managing the policies through REST APIs, you need a REST client like Postman. ## Policy Operations Application developers could refer [Resource Access Management API Specification](api-resourceaccessmanagement-api.html) for details about Policy CRUD operations. # Tenant Management Service – API Overview ## Region Europe 1 [Download OpenAPI Specification](/insights-hub/api_specs/tenantmanagement-api-xcelerator.yaml) # Tenant Management Service ## Idea Tenant Management is a central service that provides the possibility to manage an environment (tenant), its related subtenants, as well as related data. ## Access For accessing this service, you need to have the respective roles listed in [Tenant Management roles and scopes](https://documentation.mindsphere.io/MindSphere/concepts/concept-roles-scopes.html#tenant-management-service). ## Basics ### Environments An environment is an organization-specific logical environment for your data. An environment typically represents a legal entity, such as a company or corporation. In this context, Insights Hub is a multi-tenant architecture. ### Subtenants A subtenant is an organization-specific logical environment for the data of a co-operating legal entity. A subtenant typically represents a subset of the environment (tenant's) data which the tenant wants to share with one or more of its customers. ## Information of an environment (tenant) The environment information stores the following parameters: - **country**\ The home base of your legal entity - **displayName**\ The brand name which is shown in the Insights Hub environment - **name**\ This attribute specifies the tenant name. - **id**\ This is the tenant ID generated when the tenant is created and it is a 10‑digit ID. - **prefix**\ An ID which is used by developers to identify relevant code components - **type**\ The tenant type: `USER`, `DEVELOPER` or `OPERATOR` ### Company logo An environment can display its own company logo in the UI. For this purpose logos of type .jpg, .png, or .gif with a maximum file size of 1 MB can be uploaded. ### Legal Information of an environment An environment can display its own legal/branding information, which is defined as legal links of type `www`, `phone`, and `mail`. When environment branding is enabled, it automatically creates the region `global` and sets the default language to English. Environments can be located in different regions with different legislation, which might require displaying different legal links depending on their location. For this case, you can define custom regions and using country codes specify which countries belong to each region. One country can only belong to one region. You can also define different languages and add them to multiple regions. For each region, you can define individual legal links in multiple languages. The customized legal links are displayed for all users of the environment except tenant admins, who still see the original legal links of Insights Hub. The environment's location and the language settings of the web browser determine which legal links are displayed. Info You must enable tenant branding in order to create a subtenant. ## Features The Tenant Management Service exposes its API for realizing the following tasks: - Retrieve environment (tenant)-specific information - Manage your own legal information - Create, retrieve and delete subtenants ## Example Scenario ### Providing Legal Information The administrator of a machine manufacturer updates the legal information of the environment with their own company's legal information in order to provide it to other environments and subtenants. ### Creating Subtenants The administrator of a machine manufacturer wants to create a subtenant for one of their customers in order to share relevant machine data with them. For more information, refer the following : - [Use the Tenant Management API to create a Subtenant](api-tenantmanagement-api.html) - [Developing Subtenancy-Aware Applications](https://documentation.mindsphere.io/MindSphere/howto/howto-subtenant.html#using-subtenant-impersonation) - [Using subtenancy](https://documentation.mindsphere.io/MindSphere/howto/howto-subtenant.html) # Usage Transparency Service – API Overview ## Region Europe 1 [Download OpenAPI Specification](/insights-hub/api_specs/usagetransparency-v3-0-1-xcelerator.yaml) # Usage Transparency Service Usage Transparency (UTS) provides an API to get information about provider app usage and to import application usages by users/assets/tenants for billing, reporting, quota checking purposes. UTS accepts all the data that is sent in the valid format and payload does not exceed the limit of usages count per request. In case the rules are configured for certain usage units, the corresponding usage data is verified as per the configuration. If the data verification fails, the data will be rejected and is not aggregated. Users can query for the status of the data sent using `/usagesJobs` endpoint. A rule is necessary for the usages to be billed. This information can be displayed and analyzed using the [Usage Transparency UI](https://documentation.mindsphere.io/resources/html/usage-transparency-service/en-US/index.html). ## Access For accessing this service, you need to have the respective roles listed in [Usage Transparency Service roles and scopes](https://documentation.mindsphere.io/MindSphere/concepts/concept-roles-scopes.html#usage-transparency-service). ## Features Usage Transparency Service exposes its API for the following task: - Usage: Sending application specific data to the UTS backend. - Usage Jobs Traceability: Add usages of one or more tenants, users and application to UTS for storage and processing and get information about the query status, processing status and processing errors of the added usages. - Aggregated monthly Usage: Get aggregated monthly usage data for provider tenant and consumer tenant. Info Throttling for the Aggregated monthly Usage end points will be applied in the upcoming releases. UTS accepts the data only if the following checkpoints are verified: 1. A Rule has to be configured for the appname and usage unit. 1. Tenant claim in the token has to be either Developer tenant or Operator tenant. 1. Tenant claim in token and the tenantID in the Customer ID should be same or Tenant in the payload must be an IoT-tenant of an Operator and the app in the payload is provisioned to that IoT tenant. ## Limitations - The maximum usages that can be added to UTS in a single request is 200. - The maximum jobs that can be queried at a time by a user is 1000. - The amount of operations that can be performed at a time are limited. ## Terms - Rules: Rules are created in UTS to define which valid data will be accepted as usage. - KPIs: KPIs are created in UTS to define how the accepted usage data will be aggregated and made available for consumer. ## Error codes The following error codes might occur at any of the specified operations. Generic errors are prefixed with `mdsp.core.generic.`. - `204: noContent` - `400: invalidParameter` - `400: invalidRequestBodyProperty` - `400: missingParameter` - `400: missingRequestBodyProperty` - `401: unauthorized` - `403: forbidden` - `404: noMatch` - `413: payloadTooLarge` - `415: unsupportedMediaType` - `429: tooManyRequests` - `500: internalServerError` ## Example Scenario A provider offers an application with which customers can connect industrial hardware devices to Insights Hub. They want to charge an initial fee for onboarding new hardware devices and a monthly fee per device. They use the Usage Transparency within the application to send user information to the central usage service, where it can be accessed using the UI when preparing the bills. # Usage Transparency Service – Samples ## Adding usages to UTS This API can be used by operators and applications to create a job that adds application specific usages (for example, the number of created report pages, assets configured, etc.) of multiple tenants and users to UTS. Add usages to UTS using the following endpoint: ```http POST /usagesJobs ``` Sample Request: ```json { "users": [ { "tenantId": "siemenshcops", "userId": "john.doe@healthcare.siemens.com", "userType": "user", "resources": [ { "application": "manage-my-machines", "alias": "analyze-my-performance", "resource": "asset-id-1", "usages": [ { "value": 20, "unit": "mmm-recurring-unit", "datetime": "2011-07-14T19:43:37Z" } ] } ] } ] } ``` Sample Response: ```json { "id": "35a4cb01-2497-4191-ad1c-551a1942c409-20190520131", "time": "2011-07-14T19:43:37Z", "status": "ACCEPTED", "usagesCount": 7 } ``` ## Retrieving query status This API gives the overall status summary of the jobs and optionally the usages added using `/UsagesJobs` or `/usages` APIs. Description of job status is as follows: - ACCEPTED: A job is created and the usages has been accepted. - INPROGRESS: Processing is on-going for at least one of the usages added by the job. - ERRORS: There is at least one error encountered while processing the usages. - COMPLETED: All the processes applicable to the usages have been completed. Retrieve query status from UTS using the following endpoint: ```http GET /usagesJobs ``` Sample Response: ```json { "jobs": [ { "id": "35a4cb01-2497-4191-ad1c-551a1942c409-20190520131", "time": "2011-07-14T19:43:37Z", "status": "ACCEPTED", "usagesCount": 7 } ], "page": { "number": 2, "size": 10, "totalElements": 40, "totalPages": 4 } } ``` ## Retrieving query status of usages This API gives the status summary of the usages added using /UsagesJobs or /usages APIs. Description of Usages status is as follows: - ACCEPTED: Usages have been accepted. - VERIFIED: Usages have been verified against the rules configured in rule engine, if applicable. - VERIFICATIONFAILED: Verification of at least one of the usages is failed. - NOVERIFICATION: No applicable rule was found and hence no verification was done. - AGGREGATED: Usages have been aggregated into a running daily aggregate value based on the aggregation function defined in rule. In case no rule is applicable, default aggregation function 'SUM' is used. The aggregation is performed every hour and the daily aggregate is updated. Once aggregated, the UTS is queried to report the usages through UI or data endpoints. - SENTFORBILLING: If billing is enabled for the usages, the data has been sent successfully to the billing system. - FAILEDTOSENDFORBILLING: Errors were encountered while sending the data to billing. These could be network issues or other internal errors. - RECEIVEDFORBILLING: Data sent for billing is verified to be received in the billing system. Retrieve query status of the usages using the following endpoint: ```http GET /usagesJobs/{id} ``` Sample Response: ```json { "id": "35a4cb01-2497-4191-ad1c-551a1942c409-20190520131", "time": "2011-07-14T19:43:37Z", "status": "ACCEPTED", "usagesCount": 7, "usagesSummary": [ { "application": "manage-my-machine", "unit": "mm-onboarding-unit", "usagesCount": 2, "processStatus": "ACCEPTED" } ], "page": { "number": 2, "size": 10, "totalElements": 40, "totalPages": 4 } } ``` ## Retrieving processing errors This API gives the processing errors associated with the set of usages that were received by the identified job. In case of 'Verification Errors’, the list of usages that failed the verification is part of the errors. In case of 'Billing Errors’, billing ids are part of the error data. To resolve the issues, contact the support team. Retrieve processing errors for the usages using the following endpoint: ```http GET /usagesJobs/{id}/errors ``` Sample Response: ```json { "errors": [ { "code": "mdsp...", "message": "", "messageParameters": [ { "name": "name1", "value": "value1" } ], "logref": "0bff7e7a-cd25-4576-9908-4180ef086174", "errorType": "ValidationError", "info": "http://developer.mindsphere.io/uts/documentation/billing" } ], "page": { "number": 2, "size": 10, "totalElements": 40, "totalPages": 4 } } ``` # Token Management Service – API Overview ## Region Europe 1 [Download OpenAPI Specification](/insights-hub/api_specs/token-manager-api-specs-xcelerator.yaml) # Token Management Service ## Idea Use the Token Management Service to grant your application access to data of tenants, who use the application. This is relevant if your application frequently performs data processing on multiple tenants without user interaction, e.g. pre-calculating KPIs for a dashboard. If an application has access to the Token Management Service, it can request tokens to access IoT data of other tenants. The access to the Token Management Service must be explicitly granted for each version of an application. ## Access For accessing the Token Management Service, your application must explicitly be granted access using the [Developer Cockpit](https://documentation.mindsphere.io/MindSphere/system-tools/developer-cockpit.html) during development and the [Operator Cockpit](https://documentation.mindsphere.io/MindSphere/apps/operator-cockpit/introduction.html) for productive use. ## Basics Whenever IoT data is accessed, a valid token is required to get access permission. If an application requires to access IoT data without human interaction, it needs a technical token. The Token Management Service generates these tokens. In order to issue a token, the application sends a request to the Token Management Service, which contains its own [credentials](#application-credentials) and specifies which data it needs to access. If the application has permission to access this data, the Token Management Service returns a valid token, which grants the requested access permission. ### Architecture The figure below shows the basic interactions between Token Management Service and the Developer Cockpit, the Operator Cockpit, and an application. When application developers upload their applications for testing to the Developer Cockpit, it is not possible to access data on other tenants. However, the communication between the application and the Token Management Service can be tested. [1] An application requests a token from the Token Management API, if it needs to access IoT data. In its request, it provides its application credentials as well as the location it wants to access. If the application has permission to access this data, the Token Management Service returns a valid token for this action. [2] The application uses the token in order to access the requested data. [3] When an application is registered in the Operator Cockpit, it can be assigned read/write, limited, Data Contextualization read/write or custom access. Read/Write access grants full access to all available Insights Hub APIs. Limited access allows the application to manage assets and files and create events, but not to write time series data. Data Contextualization read/write access grants permission for only Data Contextualization APIs. Custom access grants access to the user selected Insights Hub APIs. If a customer purchases an application, they must confirm that the application may access their data. [4] ### Application Credentials Applications must send their application credentials to the Token Management Service in order to request a token. Application credentials consist of a client ID and a client secret. They are created after uploading an application to the [Developer Cockpit](https://documentation.mindsphere.io/MindSphere/system-tools/developer-cockpit.html) or [Operator Cockpit](https://documentation.mindsphere.io/MindSphere/apps/operator-cockpit/introduction.html) and must be provided as environment variables of the application.\ Access for an application must be issued manually in the Authorization Management in order to create application credentials. Info When you issue application credentials, the Developer Cockpit and Operator Cockpit only display the credentials once. It is not possible to review them. Application credentials are version specific and must be updated, if a new version of an application is uploaded. This also means, that the application credentials issued in the Developer Cockpit are not valid anymore, once the application has been transferred to the operator tenant. In order to revoke the credentials, de-register the application. ## Features Token Management Service exposes its API for the following task: - Issuing tokens in the name of user tenants for a specific application version. - Providing a list of user tenants authorized to access an application with pagination support. ## Limitations - This service does not issue more than 5 access tokens per request. ## Example Scenario An operator wants to offer an application on the Insights Hub Store, which automatically pre-calculates KPIs and schedules data processing for tenants who buy the application. They register their application to use the Token Management Service in order to get access to the tenants' data without requiring user action. # Token Management Service – Samples ## Generating the X-SPACE-AUTH-KEY 1. Encode the following combination of user name/ID and password/secret using Base64: ```text : ``` 1. Build the `` using the word `Basic`, followed by a space and the encoding result, e.g.: ```text X-SPACE-AUTH-KEY : Basic ZGlvcDEtaGVybWlvbmUtaGVybWlvbmU6c2RqaGZhc2RqaGZqYXNkaGZqa2FzZGhmams ``` ## Getting a Token to Access User IoT Data Use the following endpoint: ```http POST api/technicaltokenmanager/v3/oauth/token ``` Define the following header keys, replace `` with your authorization key, which is generated as explained [above](#generating-the-x-space-auth-key): ```http Content-Type: application/json X-SPACE-AUTH-KEY : ``` Request example: ```json { "appName": "application_x", "appVersion": "1.0.0", "hostTenant": "host_tenant", "userTenant": "user_tenant_1" } ``` Sample response: ```json { "access_token": "eyJhbGciOiJSUzI1NiIsImtpZCI6ImtleS1pZC0xIiwidHlwIjoiSldUIn0.eyJqdGkiOiI4NjhhOTViYmJkZGM0ZmQ4YTU3ZTM0MGMyYmI3ZWJiNiIsInN1YiI6InNuZC1jbGllbnQiLCJzY29wZSI6WyJkZXAuZG93biIsImdyLnUiLCJwdHMuc2kiLCJkZXAuYWQiLCJ0bS50LnIiLCJkZXAuZHJlZyIsImdyLmMiLCJpbS5nLmQiLCJnci5kIiwiaW0uZy5jIiwiYXZzLnZhbCIsImNkcy5yIiwibm9zZS5zZSIsInJlcC5yIiwicmVwLnN1IiwiY2RzLnciLCJjZkFwcHMubWwiLCJnci5yIiwiY3N0LnIiLCJwdWIuciIsImltLmcuciIsInJlcC5kb3duIiwicHJ2LmcudSIsInBydi5nLnIiLCJpbS5nLnUiLCJwdWIucHQiLCJhdnMuY2tzLmNwIiwiaWFtLWFjdGlvbi5jbGllbnRfY3JlZGVudGlhbHMudGVuYW50LWltcGVyc29uYXRpb24iLCJwcnYuZy5kIiwicHJ2LmcuYyIsImRlcC5yZWciLCJwdWIucGEiLCJkZXAudXBnIiwicmVwLnVwIiwicHRzLnV0IiwicHViLnN1IiwicmVwLm11IiwiY2ZBcHBzLnNvIiwiZGVwLnVyZWciLCJjZkFwcHMuc3MiLCJkZXAuYXVkIiwicmVwLnV2IiwidXRzLnN1IiwicHRzLnBjIiwicHJ2LnIiLCJjZkFwcHMuYXIiLCJyZXAuYXAiLCJwcnYuYyIsInJlcC52dSJdLCJjbGllbnRfaWQiOiJzbmQtY2xpZW50IiwiY2lkIjoic25kLWNsaWVudCIsImF6cCI6InNuZC1jbGllbnQiLCJncmFudF90eXBlIjoiY2xpZW50X2NyZWRlbnRpYWxzIiwicmV2X3NpZyI6IjZmZjVlMDkwIiwiaWF0IjoxNTQyODY0NjYzLCJleHAiOjE1NDI4NjY0NjMsImlzcyI6Imh0dHBzOi8vY29yZS5waWFtLmV1MS1pbnQubWluZHNwaGVyZS5pby9vYXV0aC90b2tlbiIsInppZCI6ImNvcmUiLCJhdWQiOlsibm9zZSIsImNzdCIsImlhbS1hY3Rpb24uY2xpZW50X2NyZWRlbnRpYWxzIiwic25kLWNsaWVudCIsInRtLnQiLCJnciIsImltLmciLCJkZXAiLCJwdHMiLCJwcnYuZyIsImNkcyIsInV0cyIsInBydiIsImF2cy5ja3MiLCJjZkFwcHMiLCJyZXAiLCJwdWIiLCJhdnMiXSwidGVuIjoiY29yZSIsInNjaGVtYXMiOlsidXJuOnNpZW1lbnM6bWluZHNwaGVyZTppYW06djEiXSwiY2F0IjoiY2xpZW50LXRva2VuOnYxIn0.eClNyplodSUU9MFJS2eM-Mc_pU2niRCDtEGZARxrq0UhseZ4DbqMwOIW4wEFqqBvNN-mdYS6XumnnFDn4IFEnJyM0DNcCzTucjqVS4RicRsa8lKFODSdQs1wO7FOETDR0_4QHFFvhB54WEsDDzlint67dhZN44nVdM2KLNJ9wkt949MWJtUZy1VrJNz-pRq_F-5Nvh6ZCA0E_DAmCEcyR0wrxY3A2QfZhYneh8VnkTPknWOtPFdpmWp7IXfNrUmiNRMI7EwY9HNTQ4GZsGkZhDdpOOrDIxZkDfTfoUgaLGtzEX8RtLUXPmE2W3e", "token_type": "bearer", "timestamp": "1559120938825", "expires_in": 1799, "scope": "cst.r uts.su im.usr.r em.rep.r tm.st.r tm.t.r agm.r iam-action.client_credentials.tenant-impersonation uts.ri asm.r atm.r uts.rc uaa.offline_token emds.ent.r asm.rep.r", "jti": "3fcf2a5e-cc76-11e7-abc4-cec278b6b50a" } ``` ## Getting Tokens to Access Multiple Users' IoT Data Use the following endpoint: ```http POST api/technicaltokenmanager/v3/oauthTokens ``` Define the following header keys, replace `` with your authorization key, which is generated as explained [above](#generating-the-x-space-auth-key): ```http Content-Type: application/json X-SPACE-AUTH-KEY : ``` Request example: ```json { "appName": "application_x", "appVersion": "1.0.0", "hostTenantId": "host_tenant", "userTenantIds": [ "user_tenant_a", "user_tenant_b" ] } ``` Sample response: ```json { "oauthTokens": [ { "userTenantId": "user_tenant_a", "token": { "access_token": "eyJhbGciOiJSUzI1NiIsImprdSI6Imh0dHBzOi8vZGJkZTEubG9jYWxob3N0OjgwODAvdWFhL3Rva2VuX2tleXMiLCJraWQiOiJrZXktaWQtMiIsInR5cCI6IkpXVCJ9.eyJqdGkiOiJjMTk4MTIxMjY4MWY0NmM0YmNiYzNhYWI1MDc2NGYzMCIsInN1YiI6ImRiZGUxLXFldGttMS0xLjAuMDU1Iiwic2NvcGUiOlsibWRzcDpjb3JlOkFkbWluM3JkUGFydHlUZWNoVXNlciJdLCJjbGllbnRfaWQiOiJkYmRlMS1xZXRrbTEtMS4wLjA1NSIsImNpZCI6ImRiZGUxLXFldGttMS0xLjAuMDU1IiwiYXpwIjoiZGJkZTEtcWV0a20xLTEuMC4wNTUiLCJncmFudF90eXBlIjoiY2xpZW50X2NyZWRlbnRpYWxzIiwicmV2X3NpZyI6Ijg5YTFhZDNkIiwiaWF0IjoxNTU5MDUyNTg0LCJleHAiOjE1NTkwNTQzODQsImlzcyI6Imh0dHBzOi8vZGJkZTEucGlhbS5ldTEtYi5taW5kc3BoZXJlLmlvL29hdXRoL3Rva2VuIiwiemlkIjoiZGJkZTEiLCJhdWQiOlsiZGJkZTEtcWV0a20xLTEuMC4wNTUiXSwidGVuIjoiZGJkZTEiLCJzY2hlbWFzIjpbInVybjpzaWVtZW5zOm1pbmRzcGhlcmU6aWFtOnYxIl0sImNhdCI6ImNsaWVudC10b2tlbjp2MSJ9.zSrnv3ypC9gHPUNGlbAVGxA8tEoGwnOVd2Vk5XNF-XEpf34Fh2JUrG9oYUcyBPeB1pUwOvxxrGuYAFwYk1eGmdAxT0KPL7R2JTbDRPgEPA0hLZN9mw5FL-CTlifzK1isEN_6ePH9y0T2tWCHiUCL5EURcrwrGfP3Xot7Lu2g9ZR-q-ychshsH0HVIZ9GerwRGi5ciO-FI2z8z7omVPojimCbLooLE7V6Kv2mtM5lqStaANxbV1h1ITkiXkEgOpEIRHG6nwqG2LwQybTAIj9MRW-g620qB9PYDYxFcGda", "token_type": "bearer", "timestamp": "1559120938825", "expires_in": 1799, "scope": "cst.r uts.su im.usr.r em.rep.r tm.st.r tm.t.r agm.r iam-action.client_credentials.tenant-impersonation uts.ri asm.r atm.r uts.rc uaa.offline_token emds.ent.r asm.rep.r", "jti": "3fcf2a5e-cc76-11e7-abc4-cec278b6b50a" } }{ "userTenantId": "user_tenant_b", "token": { "access_token": "eyJhbGciOiJSUzI1NiIsImprdSI6Imh0dHBzOi8vZGJkZTEubG9jYWxob3N0OjgwODAvdWFhL3Rva2VuX2tleXMiLCJraWQiOiJrZXktaWQtMiIsInR5cCI6IkpXVCJ9.eyJqdGkiOiJjMTk4MTIxMjY4MWY0NmM0YmNiYzNhYWI1MDc2NGYzMCIsInN1YiI6ImRiZGUxLXFldGttMS0xLjAuMDU1Iiwic2NvcGUiOlsibWRzcDpjb3JlOkFkbWluM3JkUGFydHlUZWNoVXNlciJdLCJjbGllbnRfaWQiOiJkYmRlMS1xZXRrbTEtMS4wLjA1NSIsImNpZCI6ImRiZGUxLXFldGttMS0xLjAuMDU1IiwiYXpwIjoiZGJkZTEtcWV0a20xLTEuMC4wNTUiLCJncmFudF90eXBlIjoiY2xpZW50X2NyZWRlbnRpYWxzIiwicmV2X3NpZyI6Ijg5YTFhZDNkIiwiaWF0IjoxNTU5MDUyNTg0LCJleHAiOjE1NTkwNTQzODQsImlzcyI6Imh0dHBzOi8vZGJkZTEucGlhbS5ldTEtYi5taW5kc3BoZXJlLmlvL29hdXRoL3Rva2VuIiwiemlkIjoiZGJkZTEiLCJhdWQiOlsiZGJkZTEtcWV0a20xLTEuMC4wNTUiXSwidGVuIjoiZGJkZTEiLCJzY2hlbWFzIjpbInVybjpzaWVtZW5zOm1pbmRzcGhlcmU6aWFtOnYxIl0sImNhdCI6ImNsaWVudC10b2tlbjp2MSJ9.zSrnv3ypC9gHPUNGlbAVGxA8tEoGwnOVd2Vk5XNF-XEpf34Fh2JUrG9oYUcyBPeB1pUwOvxxrGuYAFwYk1eGmdAxT0KPL7R2JTbDRPgEPA0hLZN9mw5FL-CTlifzK1isEN_6ePH9y0T2tWCHiUCL5EURcrwrGfP3Xot7Lu2g9ZR-q-ychshsH0HVIZ9GerwRGi5ciO-FI2z8z7omVPojimCbLooLE7V6Kv2mtM5lqStaANxbV1h1ITkiXkEgOpEIRHG6nwqG2LwQybTAIj9MRW-g620qB9PYDYxFcGdb", "token_type": "bearer", "timestamp": "1559120938828", "expires_in": 1799, "scope": "cst.r uts.su im.usr.r em.rep.r tm.st.r tm.t.r agm.r iam-action.client_credentials.tenant-impersonation uts.ri asm.r atm.r uts.rc uaa.offline_token emds.ent.r asm.rep.r", "jti": "3fcf2a5e-cc76-11e7-abc4-cec278b6b50b" } } ] } ``` ## Getting a List of all Authorized Users of an Application Use the following endpoint: ```http GET api/technicaltokenmanager/v3/userTenants ``` Info This endpoint expects a token in the authorization header with bearer scheme. The token can be obtained from `/oauth/token` using the operator tenant as `userTenant` and `hostTenant`. Sample response: ```json { "page": { "size": 0, "totalElements": 0, "totalPages": 0, "number": 0 }, "userTenants": [ { "id": "testusertenant1" } ] } ``` Info This endpoint only returns up to 100 tenant IDs per request. ## Getting Tokens to Access all Users' IoT Data 1. Request a token from the `/oauth/token` endpoint as described in [Getting a Token to Access User IoT Data](#getting-a-token-to-access-user-iot-data) using the tenant where your app runs on as `userTenant` and `hostTenant`. Request example: ```json { "appName": "application_x", "appVersion": "1.0.0", "hostTenant": "operator_tenant", "userTenant": "operator_tenant" } ``` 1. Get a list of all author tenants from the `/userTenants` endpoint as described in [Getting a List of all Authorized Users of an Application](#getting-a-list-of-all-authorized-users-of-an-application). Info This endpoint only returns up to 100 tenant IDs per request. 1. Request up to 5 access tokens from the `/oauthTokens` endpoint as described in [Getting Tokens to Access Multiple Users' IoT Data](#getting-tokens-to-access-multiple-users-iot-data). Repeat this step if required to get access tokens for all user tenants. Request example: ```json { "appName": "application_x", "appVersion": "1.0.0", "hostTenantId": "host_tenant", "userTenantIds": [ "user_tenant_a", "user_tenant_b" ] } ``` ## Best Practices to Issue Tokens - Cache tokens and only issue a new one if they expire.\ Although the Token Manager API provides caching, it is recommended to implement your own caching to overcome the network latency on the request round trip. - Do not expose tokens via endpoints. - Do not print the service credentials in the application log. # Integrated Data Lake Service – API Overview Region Europe 1 [Download OpenAPI Specification](https://documentation.mindsphere.io/insights-hub/api_specs/integrated-data-lake-v3-14-0-eu1-xcelerator.yaml) # Object and Folder Operations ## Object Operations via App Gateway The user can upload the files and the associated metadata separately with a single API call. To upload the file, user needs to generate pre-signed URL i.e. SAS URL. It is possible to use multiple times during the validity period, minimum of 15 minutes. The requirement is to eliminate the use of pre-signed URL with prolonged validity periods. Uploading and downloading the files using APIs will provide additional flexibility in handling other scenarios like secure lock and managing object lifecycle. This API endpoint allows the user to upload an object to a given path along with the metadata. The validations on the metadata is done as per the configurations done by the Integrated Data Lake Administrator. Validations on the metadata is done and the file upload is completed. Once the file upload is completed, the metadata is saved. If the file upload is complete and the metadata update fails, then file is only visible to Integrated Data Lake Administrator if the Attribute Based Access Control (ABAC) rules do not allow reading of the file without the object metadata. Object operations via app gateway are described below: - **Upload an object via app gateway with metadata** APIs can upload the files up to 100MB, multipart upload is provided later. In the API endpoint, if the path contains special characters, they are encoded. Encode only the words that contains special characters. Special characters supported in file and folder name are described below: - Alphanumeric characters (word characters: A-Z, a-z, 0-9 and underscore"\_") - Space (" "), except at the start and end - Period (".") - Exclamation mark ("!") - Asterisk ("\*") - Single quote ("'") - Ampersand ("&") - Dollar sign ("$") - At symbol ("@") - Colon (":") - Plus sign ("+") - Comma (",") - Square brackets ("[" and "]") - Less than ("\<") and greater than (">") symbols - Tilde ("~") - Double quote (""") - Hash ("#") - Vertical bar ("|") - Question mark ("?") - Curly braces ("{" and "}") - Caret ("^") - Parentheses ("(" and ")") - Underscore ("\_") but not at the beginning or at end, only for folder name - Hyphen ("-") - Equal sign ("=") - **Download Object via App Gateway** Attribute Based Access Control (ABAC) is used to download an object in the path is permitted as per the policy created for the user by tenant admin. For more information on ABAC, refer to [Attribute Based Access Control (ABAC) configuration](api-integrated-data-lake-samples-metadata-management.html#attribute-based-access-control). ## Folder Operations via App Gateway The user can create the folders and the associate metadata with a single API call. This API endpoint allows the user to create a folder in a given path along with the metadata. The validations on the metadata is done as per the configurations. After the validation on the metadata is done, a folder is successfully created. Once the folder creation is completed, the metadata is saved. If a folder creation is completed and the metadata update fails, then a folder will only be visible to the Integrated Data Lake Administrator, if the Attribute Based Access Control (ABAC) rules do not allow access on a folder without the metadata. - **Create empty folder with Metadata** User can create an empty folder for the specified path and optional metadata values as per the metadata keys defined by the Integrated Data Lake admin. Note If folder names contains special characters, no encoding is required as it is part of request body. Special characters supported in the folder name are described below: - Alphanumeric characters (word characters: A-Z, a-z, 0-9, and underscore "\_") - Space (" "), except at the start and end - Period (".") - Exclamation mark ("!") - Asterisk ("\*") - Single quote ("'") - Ampersand ("&") - Dollar sign ("$") - At symbol ("@") - Colon (":") - Plus sign ("+") - Comma (",") - Square brackets ("[" and "]") - Less than ("\<") and greater than (">") symbols - Tilde ("~") - Double quote (""") - Hash ("#") - Vertical bar ("|") - Question mark ("?") - Curly braces ("{" and "}") - Caret ("^") - Parentheses ("(" and ")") - Hyphen ("-") - Equal sign ("=") - **Delete an empty Folder** Delete an empty folder along with the metadata key values from the specified path. Note Operation is performed only on leaf folder. There is no impact on folder hierarchy. # Integrated Data Lake Service Note Integrated Data Lake Service API version 4.*.* is available only for Virtual Private Cloud. ## Idea The Integrated Data Lake (IDL) is a repository that allows you to store structured and unstructured data in its native format as long as it is required. It handles large data pools for which the schema and data requirements are not defined until the data is queried. This offers more agility and flexibility than the traditional data management systems. The Integrated Data Lake Service allows you to store your data as it is, analyze it using dashboards and visualizations or use it for big data processing, real-time analytics, and machine learning. ## Access For accessing Integrated Data Lake Service, you need to have the respective roles listed in [Data lake Services roles and scopes](https://documentation.mindsphere.io/MindSphere/concepts/concept-roles-scopes.html#integrated-data-lake). A user can only interact with objects within their environment and subtenants. For accessing the Secure Data Sharing (SDS) protected APIs, you need to have the appropriate Policy Definitions in place. For the list of supported APIs and the required actions, refer to [Integrated Data Lake Service](../core-resourceaccessmanagement/api-resourceaccessmanagement-services-actions-map.html#integrated-data-lake-service). Note If your organization's security policy includes outbound traffic restrictions enforced by a firewall, whitelist the following URLs to ensure seamless access.\ Integrated Data Lake: `datalake-prod--.s3.eu-central-1.amazonaws.com`\ CNAME: `s3-r-w.eu-central-1.amazonaws.com` ## Basics ### Data Upload and Download The Integrated Data Lake Service enables data upload and download using AWS, Azure or Aliyun. The signed URLs for AWS and Aliyun and Shared Access Signatures (SAS) for Azure have an expiration date and time and can only be used by authorized environment users or services. | Data upload and download | AWS | Azure | Aliyun | | ------------------------- | --------------- | ----------------------------- | --------------- | | Using service | Pre-signed URLs | Shared Access Signature (SAS) | Pre-signed URLs | | Maximum object size limit | 5 GB | 256 MB | 5 GB | ### Time Series Import The Integrated Data Lake Service allows authorized environment users or services to import time series data into the data lake. This enables on-demand time series upload for analytics and machine learning tools. ### Metadata Management Metadata management is a crucial aspect of data management within any Data Lake application. It involves systematic organization, storage, retrieval, and maintenance of metadata, which is essentially data about data. This includes information about the characteristics, origin, usage, and relationships of the actual data stored in the Data Lake. The Integrated Data Lake Service assigns each object a unique identifier. Additionally, the object can be assigned a set of extended metadata values. #### Metadata Collections Metadata collection is a systematic process of collecting metadata keys that manage all the relevant metadata information associated with a specific dataset or system. #### Metadata Keys Metadata keys are unique identifiers that are used to represent specific attributes or characteristics associated with the data in a key-value pair structure. #### Metadata Rules Metadata rules are a set of pre-defined conditions that determines the metadata tied to a custom collection that is applicable to Integrated Data Lake resources. ### Data Access Using these services, you can enable (and disable) read and write access to your data for a specific environment. For example, you can enable analytics tools to directly access your data for analysis without having to download it. This saves storage space and eliminates the need for regular data synchronization. Alternatively, the Integrated Data Lake Service can generate temporary read and write access to your data. For more information, refer the below table: | Deviation | AWS | Access Permission | Azure | Access Permission | Aliyun | Access Permission | | -------------------- | -------------------------- | ----------------- | -------------------- | ----------------- | ---------------------------- | ----------------- | | Data access services | Cross account access | READ | Service Principal | READ and WRITE | Cross account access | READ | | | Simple Token Service (STS) | READ, WRITE | Pre-signed URL | READ, WRITE | Security Token Service (STS) | READ, WRITE | | Access limits | 5 cross account accesses | | 5 Service Principals | | 5 cross account accesses | | The Security Token Service or Service Principal can only be used by an authorized environment user or service. ### Notification The Integrated Data Lake Service provides a notification functionality, which notifies whenever objects are ingested, updated or deleted using the service. Only the authorized environment users or services can subscribe to notifications. Currently, a maximum of 15 notifications can be subscribed. Info If the permission to send notification to Simple Notification Service (SNS) topic is removed, then the tenantAdmin will be notified through an email to check and respond. The email will be sent for a week and thereafter the subscription will be inactivated. ### Event Subscription Event Subscription allows you to subscribe to the events to get notifications. You can register with a Simple Notification Service (SNS) destination path on AWS or Service Bus destination path on Azure for notifications which will be published by Integrated Data Lake service. These notifications include object events like create, update or delete in environment prefix. You can add, view, edit and delete the event subscriptions. If the permission to send notification to a SNS/service bus topic is removed or if there is any misconfiguration, then the TenantAdmin will be notified through an email. The status of the subscription will be changed to "Inactive" in the user interface. You can change the subscription status to "Active" in the user interface after resolving the configuration. Info Subscription name is not mandatory. If the name is not provided, then (subscription\_\<<1234>>) will be considered as the default name for the subscription. ## Features Integrated Data Lake services exposes its API for realizing the following tasks: - Import time series data - Generate signed URLs or to upload, update or download objects - Delete objects - Add, update and delete metadata values for objects - Receive notifications - Cross account access for AWS and Aliyun/Service Principal (for Azure) - Subtenancy support - Bulk batch upload of objects The Integrated Data Lake UI provides the following functionalities: - Cross account access (on AWS) and (on Aliyun)/Service Principal (on Azure) - For enabling native account to read the data from IDL. - TimeSeries Import functionality - To import time series data into Data Lake - Data Explorer - For enabling to explore the files/objects. - Event Subscription - For creating subscription for events on data. - OData - For creating contracts for objects on data. - Metadata Management- For systematic organization, storage, retrieval, and maintenance of metadata, which is essentially data about data. ## Limitations - All requests pass through [Industrial IoT Gateway](https://documentation.mindsphere.io/MindSphere/concepts/concept-gateway-url-schemas.html) and must adhere to the [Industrial IoT Gateway Restrictions](https://documentation.mindsphere.io/MindSphere/concepts/concept-gateway-url-schemas.html#restrictions). - Maximum supported object size for object upload and download using signed URL is 5 GB. - Maximum supported object size for object upload and download using Shared Access Signatures (SAS) is 256 MB. - Signed URLs expire after two hours. - Shared Access Signatures (SAS) expire after twelve hours. - Objects are not version controlled. - The cross account accesses will be emptied and stopped working as expected under the revocation process as per the bucket policy. - The token will remain active until its expiry before deprovisioning. - The S3 Signed URL will remain active until its expiry before deprovisioning. - All the Bulk Import limitation will still be valid for time series import functionality in IDL. - A maximum of 10 cross account accesses can be created in disabled state for AWS. - A maximum of 5 cross account accesses can be enabled for any given time for AWS and for Aliyun. - A maximum of 5 service principals can be enabled for any given time for Azure. - A user can subscribe to only 15 subscriptions. - The data in UTS might take 48hrs to reflect. - "Write" access cannot be provided at Time Series Import folder in Service Principal. - Upload Pre-Signed URL in AWS and Shared Access Signatures (SAS) in Azure for Time Series Import folder cannot be created. - User path should be pre-fixed with Time Series Import for downloading the time series data files. - Characters used for values of the file name must be matching with the regex pattern character set '[a-zA-Z0-9.!\*'() \_-/=]'. Spaces are not allowed in the beginning or at the end. Also, consecutive spaces are not allowed within the name. - Objects uploaded by using native URLs will be deleted by using native URLs only. IDL Service URLs do not support the deletion of files which are uploaded using native URLs. - A maximum of 2 secrets can be generated at any given time for each Service Principal. - Secret will be active for maximum 90 days, thereafter it will be expire automatically. - For event notification, user should provide the topic from EU1 region only. Integrated Data Lake will not be able to send the notification to other region topics. - It will take approximately 5-10 minutes for the data to be available in the search, after uploading in Integrated Data Lake. - OData is supported for EU1 (AWS) region only. To get the current list of limitations refer to [release notes](https://documentation.mindsphere.io/resources/html/release-notes/en-US/index.html). ## Example Scenario The quality assurance representative of an airline company wants to upload flight data (years 2009-2019) to Industrial IoT. So, they can run analytics tools and make the data accessible for querying. They can use the Integrated Data Lake Service to upload Excel sheets and enable data access from other accounts. This allows the airline company to integrate analytics tools like AWS Glue or Power BI on Azure and quickly perform queries. For example, they can query for "the most popular airport in last 10 years" or "the airport with most cancelled flights in the past year". # Integrated Data Lake Service ## Idea The Integrated Data Lake (IDL) is a repository that allows you to store structured and unstructured data in its native format until it is needed. It handles large data pools for which the schema and data requirements are not defined until the data is queried. This offers more agility and flexibility than traditional data management systems. The Integrated Data Lake Service allows you to store your data as is, analyze it using dashboards and visualizations or use it for big data processing, real-time analytics, and machine learning. ## Access For accessing Integrated Data Lake Service, you need to have the respective roles listed in [Data lake Services roles and scopes](https://documentation.mindsphere.io/MindSphere/concepts/concept-roles-scopes.html#integrated-data-lake). A user can only interact with objects within their environment and subtenants. For accessing the Secure Data Sharing (SDS) protected APIs, you need to have appropriate Policy Definitions in place. For the list of supported APIs and required actions, refer to [Integrated Data Lake Service](../core-resourceaccessmanagement/api-resourceaccessmanagement-services-actions-map.html#integrated-data-lake-service). ## Basics ### Data Upload and Download The Integrated Data Lake Service enables data upload and download using AWS, Azure or Aliyun. The signed URLs for AWS and Aliyun and Shared Access Signatures (SAS) for Azure have an expiration date and time and can only be used by authorized environment users or services. | Data upload and download | AWS | Azure | Aliyun | | ------------------------- | --------------- | ----------------------------- | --------------- | | Using service | Pre-signed URLs | Shared Access Signature (SAS) | Pre-signed URLs | | Maximum object size limit | 5 GB | 256 MB | 5 GB | ### Time Series Import The Integrated Data Lake Service allows authorized environment users or services to import time series data into the data lake. This enables on-demand time series upload for analytics and machine learning tools. ### Metadata The Integrated Data Lake Service assigns each object a unique identifier. Additionally, the object can be assigned a set of extended metadata tags. ### Data Access Using these services, you can enable/disable read and write access to your data for a specific environment. For example, you can enable analytics tools to directly access your data for analyses without having to download it. This saves storage space and eliminates the need for regular data synchronization. Alternatively, the Integrated Data Lake Service can generate temporary read and write access to your data. Refer the below table: | Deviation | AWS | Access Permission | Azure | Access Permission | Aliyun | Access Permission | | -------------------- | -------------------------- | ----------------- | -------------------- | ----------------- | ---------------------------- | ----------------- | | Data access services | Cross account access | READ | Service Principal | READ and WRITE | Cross account access | READ | | | Simple Token Service (STS) | READ, WRITE | Pre-signed URL | READ, WRITE | Security Token Service (STS) | READ, WRITE | | Access limits | 5 cross account accesses | | 5 Service Principals | | 5 cross account accesses | | The Security Token Service or Service Principal can only be used by an authorized environment user or service. **For AWS only**: For example, IDL user has third-party application enabled on the AWS account, say tableau server. Now, the user wants to give access to the tableau server to the data that resides in IDL. This can be easily done by enabling the AWS account using cross account access and performing the desired use case. Further, it is also possible to provide read and delete access to the enabled cross account through API or through the IDL manager. ### Notification The Integrated Data Lake Service provides a notification functionality, which reports when objects are ingested, updated or deleted using the service. Authorized environment users or services can subscribe to notifications. Currently, only environment user or services can subscribe to only 15 notifications. Info If the permission to send notification to Simple Notification Service (SNS) topic is removed, then the tenantAdmin will be notified through an email to check and respond. The email will be sent for a week and thereafter the subscription will be inactivated. ### Event Subscription Event Subscription allows you to subscribe to the events to get notifications. You can register with a Simple Notification Service (SNS) destination path on AWS or Service Bus destination path on Azure for notifications which will be published by Integrated Data Lake service. These notifications include object events like create, update or delete in environment prefix. You can add, view, edit and delete the event subscriptions. If the permission to send notification to a SNS/service bus topic is removed or if there is any misconfiguration, then the TenantAdmin will be notified through an email. The status of the subscription will be changed to "Inactive" in the user interface. You can change the subscription status to "Active" in the user interface after resolving the configuration. Info Subscription name is not mandatory. If the name is not provided, then (subscription\_\<<1234>>) will be considered as the default name for the subscription. ## Features Integrated Data Lake services exposes its API for realizing the following tasks: - Import time series data - Generate signed URLs or to upload, update or download objects - Delete objects - Add, update and delete tags for objects - Receive notifications - Cross account access for AWS and Aliyun / Service Principal (for Azure) - Subtenancy support - Bulk batch upload of objects The data lake services exposes UI for below functionalities: - Cross account access (on AWS) and (on Aliyun) / Service Principal (on Azure) - For enabling native account to read the data from IDL. - TimeSeries Import functionality - To import time series data into Data Lake - Data Explorer - For enabling to explore the files/objects. - Event Subscription - For creating subscription for events on data. - OData - For creating contracts for objects on data. ## Limitations - All requests pass through [Industrial IoT Gateway](https://documentation.mindsphere.io/MindSphere/concepts/concept-gateway-url-schemas.html) and must adhere to the [Industrial IoT Gateway Restrictions](https://documentation.mindsphere.io/MindSphere/concepts/concept-gateway-url-schemas.html#restrictions). - Maximum supported object size for object upload and download using signed URL is 5 GB. - Maximum supported object size for object upload and download using Shared Access Signatures (SAS) is 256 MB. - Signed URLs expire after two hours. - Shared Access Signatures (SAS) expire after twelve hours. - Objects are not version controlled. - The cross account accesses will be emptied and stopped working as expected under the revocation process as per the bucket policy. - The token will remain active until its expiry before deprovisioning. - The S3 Signed URL will remain active until its expiry before deprovisioning. - All the Bulk Import limitation will still be valid for time series import functionality in IDL. - A maximum of 10 cross account accesses can be created in disabled state for AWS. - A maximum of 5 cross account accesses can be enabled for any given time for AWS and for Aliyun. - A maximum of 5 service principals can be enabled for any given time for Azure. - A user can subscribe to only 15 subscriptions. - The data in UTS might take 48hrs to reflect. - "Write" access cannot be provided at Time Series Import folder in Service Principal. - Upload Pre-Signed URL in AWS and Shared Access Signatures (SAS) in Azure for Time Series Import folder cannot be created. - User path should be pre-fixed with Time Series Import for downloading the time series data files. - Characters used for values of the file name must be matching with the regex pattern character set '[a-zA-Z0-9.!\*'() \_-/=]'. Spaces are not allowed in the beginning or at the end. Also, consecutive spaces are not allowed within the name. - Objects uploaded by using native URLs will be deleted by using native URLs only. IDL Service URLs do not support the deletion of files which are uploaded using native URLs. - A maximum of 2 secrets can be generated at any given time for each Service Principal. - Secret will be active for maximum 90 days, thereafter it will be expire automatically. - For event notification, user should provide the topic from EU1 region only. Integrated Data Lake will not be able to send the notification to other region topics. - It will take approximately 5-10 minutes for the data to be available in the search, after uploading in Integrated Data Lake. - OData is supported for EU1 (AWS) region only. To get the current list of limitations, navigate to [release notes](https://documentation.mindsphere.io/resources/html/release-notes/en-US/index.html), choose the latest date and refer to the required IoT service. ## Example Scenario The quality assurance representative of an airline company wants to upload flight data (years 2009-2019) to Industrial IoT. So, they can run analytics tools and make the data accessible for querying. They can use the Integrated Data Lake Service to upload Excel sheets and enable data access from other accounts. This allows the airline company to integrate analytics tools like AWS Glue or Power BI on Azure and quickly perform queries. For example, they can query for "the most popular airport in last 10 years" or "the airport with most cancelled flights in the past year". # Bulk Batch File Upload Note This section is applicable only for region Europe 1. Bulk batch upload of objects in Integrated Data Lake adds the possibility to import or export data in bulk. It can be done from and to the external servers or programs to analyze or filter massive data. Bulk batch file upload uses the STS token provided by AWS with write access. It is required to specify the folder in which the bulk files need to be uploaded. Thereafter, the STS token with write access to that path (folder) is generated. Using this STS token, you can upload files in bulk. The validity for the STS token is up to 12 hours. ## Set environment variables Set the environment variables for your system: Environment Paths on Windows C:> setx AWS_ACCESS_KEY_ID XXXXXXXXXXXXXXXXXXXX C:> setx AWS_SECRET_ACCESS_KEY xxxxxxxxxxxxx/xxxxxxx/xxxxxxxxxxxxxxxxxx C:> setx AWS_SESSION_TOKEN=\<> Environment Paths on Linux $ export AWS_ACCESS_KEY_ID=XXXXXXXXXXXXXXXXXXXX $ export AWS_SECRET_ACCESS_KEY=xxxxxxxxxxxxx/xxxxxxx/xxxxxxxxxxxxxxxxxx $ export AWS_SESSION_TOKEN=\<> ## Provide write permission The tenantAdmin can access this API and provide write permission on folder or path, on the subtenants folder and on the root folder, except for the Time Series Import (TSI) folder. ```http POST /accessTokenPermissions ``` ```http Content-Type: Content-Type: application/json ``` **Request example:** ```json { "subtenantId": "204a896c-a23a-11e9-a2a3-2a2ae2dbcce4", "path": "/myfolder1/myfolder2", "permission": "WRITE" } ``` **Response example:** ```json { "subtenantId": "204a896c-a23a-11e9-a2a3-2a2ae2dbcce4", "id": "string", "path": "/myfolder1/myfolder2", "permission": "WRITE", "created": "2019-11-12T13:36:00.000Z" } ``` ## Get a list of write permission folders The tenantAdmin can get a list of all the folders that have the write permission. It is also possible to get a list of write permissions of the subtenants folders. ```http GET /accessTokenPermissions ``` ```http Content-Type: application/json ``` **Response example:** ```json { "accessTokenPermissions": [ { "subtenantId": "204a896c-a23a-11e9-a2a3-2a2ae2dbcce4", "id": "string", "path": "/myfolder1/myfolder2", "permission": "WRITE", "created": "2019-11-12T13:36:00.000Z" } ] } ``` ## Get the list for write permission folders for the given id You can get the list of folders that have the write permissions, for the given id. The tenantAdmin can also get the details of the request for subtenants. The subtenants can access this API to get details of the request belongs to their write folder. ```http GET /accessTokenPermissions/{id} ``` ```http Content-Type: application/json ``` **Response example:** ```json { "subtenantId": "204a896c-a23a-11e9-a2a3-2a2ae2dbcce4", "id": "string", "path": "/myfolder1/myfolder2", "permission": "WRITE", "created": "2019-11-12T13:36:00.000Z""id": "0234sd34a23a-11e9-a2a3-2a2sdfw34ce4", } ``` ## Revoke write permission The TenantAdmin can access this API and provide delete permission on folders for the given id. ```http DELETE /accessTokenPermissions/{id} ``` ```http Content-Type: application/json ``` **Response example:** ```http Status: 204 - deleted ``` ## Get the STS token with Write access Request URL: ```http https://gateway.eu1.mindsphere.io/api/datalake/v3/generateAccessToken ``` **Request example:** ```json { "durationSeconds": 900, "permission": "WRITE" } ``` **Response example:** ```json { "storageAccount": "dlbucketname", "storagePath": "data/ten=tenantname OR data/sub=subtenantId", "credentials": { "secretAccessKey": "AbCdEfGhIjKlMnOpQrStUvWxYz", "accessKeyId": "AbCdEfGhIjKlMnOpQrStUvWxYzAbCdEfGhIjKlMnOpQrStUvWxYz", "sessionToken": "AbCdEfGhIjKlMnOpQrStUvWxYzAbCdEfGhIjKlMnOpQrStUvWxYzAbCdEfGhIjKlMnOpQrStUvWxYzAbCdEfGhIjKlMnOpQrStUvWxYzAbCdEfGhIjKlMnOpQrStUvWxYzAbCdEfGhIjKlMnOpQrStUvWxYz" }, "durationSeconds": 900, "permission": "WRITE" } ``` ### Command and Output The following command is used for Bulk upload: ```http aws s3 cp s3://ms-coldstorage-iot-integ-eu-central-1/000013bb5ed740f9950504854367e12a s3://estbuckettimeseries/ten=dide2/000013bb5ed740f9950504854367e12a --recursive Completed 1.7 KiB/1.7 KiB (812 Bytes/s) with 1 file(s) remaining copy: s3://ms-coldstorage-iot-integ-eu-central-1/000013bb5ed740f9950504854367e12a/propSetName=SPSNFRTestFields/date=2019-04-05/0900.parquet to s3://estbuckettimeseries/ten=dide2/000013bb5ed740f995050485436 7e12a/propSetName=SPSNFRTestFields/date=2019-04-05/0900.parquet ``` Note The time required to upload files is dependent on the network bandwidth and location. ## STS Token Additional API Allows users to request temporary, limited-privilege AWS credentials to get read-only or write-only access on the URI returned in the response. - Generated access credentials can be used to access all the paths provided in the request with their respective permissions. - Path field is mandatory if any permission is provided in the request or vice-versa. - **LIST** permission is independent of the path and permission fields in the request. By default, **LIST** objects permission will be provided on the root prefix ("/") of environment or subtenant. - If **path and permission**, both fields are not provided in the request then “READ” permission will be given on the root prefix ("/") of environment or subtenant. - An access credential requested for a given path also automatically gives access to all sub-paths of the requested path. For example, if an access token is requested for path “/a” and there are sub-paths “/a/b” and “/a/b/c”, the token allows to access those too. - 1024 characters are supported as maximum length for path field. If the path length is shorter, then multiple accesses on different prefixes are supported in the same request. Otherwise, a single path prefix of max 1024 characters will only be supported. The number of prefixes supported will depend on the path length/permission specified in the request. ```http POST /generateAccessCredentials ``` ```http Content-Type: application/json ``` **Request example:** ```json { "durationSeconds": 900, "accesses": [ { "path": "/myfolder1/folder2", "subtenantId": "204a896c-a23a-11e9-a2a3-2a2ae2dbcce4", "permissions": [ "READ", "WRITE", "DELETE" ] } ] } ``` **Response example:** ```json { "credentials": { "secretAccessKey": "AbCdEfGhIjKlMnOpQrStUvWxYz", "accessKeyId": "AbCdEfGhIjKlMnOpQrStUvWxYzAbCdEfGhIjKlMnOpQrStUvWxYz", "sessionToken": "AbCdEfGhIjKlMnOpQrStUvWxYzAbCdEfGhIjKlMnOpQrStUvWxYzAbCdEfGhIjKlMnOpQrStUvWxYzAbCdEfGhIjKlMnOpQrStUvWxYzAbCdEfGhIjKlMnOpQrStUvWxYzAbCdEfGhIjKlMnOpQrStUvWxYz" }, "storageAccount": "dlbucketname", "durationSeconds": 900, "accesses": [ { "path": "/myfolder1/folder2", "subtenantId": "204a896c-a23a-11e9-a2a3-2a2ae2dbcce4", "permissions": [ "READ", "WRITE", "DELETE" ] } ] } ``` # Configuration of Events You can choose to be notified for object events (add, update or delete) for your space in Data Lake. It allows you to subscribe for event notifications generated when the objects of a environment or subtenant are created, updated or deleted. Multiple subscriptions for the same path can be created when each has a different destination. Similarly, multiple subscriptions for the same destination can be created when each has a different path. Maximum 15 subscriptions can be created for a environment or subtenant. Path in request payload should be upto folders and not upto object e.g. “myfolder/mysubfolder” ## Subscribe to the event notification **To create Event Subscription** - Once this step is performed, IDL will subscribe to the intended Simple Notification Service (SNS) topic. ```http POST /objectEventSubscriptions ``` **Content-Type:** application/json **Request example:** ```json { "path": "myfolder/mysubfolder", "destination": "aws-sns://arn:aws:sns:region:account-id:topicname", "subtenantId": "204a896c-a23a-11e9-a2a3-2a2ae2dbcce4" } ``` **Response example:** ````text ``` json { "id": "7dc53df5703e49b38670b1c468f47f1f", "storageAccount": "dlbucketname", "storagePath": "data/sub=subtenantId/myfolder/mysubfolder", "destination": "aws-sns://arn:aws:sns:region:account-id:topicname", "eTag": 1, "subtenantId": "204a896c-a23a-11e9-a2a3-2a2ae2dbcce4" } ```` ```` ## Get the details of events subscribed You can get the details of SNS topics to which you are already subscribed. It will list object event subscriptions for the environment or subtenant. If requester is environment, all the subscriptions for the environment as well as its all subtenants are returned. If requester is a subtenant, all the subscriptions for the subtenant are returned. If environment wants to filter results for a particular subtenant, filter query parameter `subtenantId` can be used. This filter query parameter is applicable only if the requester is environment. ```http GET /objectEventSubscriptions/ ```` **Response example:** `json { "id": "7dc53df5703e49b38670b1c468f47f1f", "storageAccount": "dlbucketname", "storagePath": "data/sub=subtenantId/myfolder/mysubfolder", "destination": "aws-sns://arn:aws:sns:region:account-id:topicname", "eTag": 1, "subtenantId": "204a896c-a23a-11e9-a2a3-2a2ae2dbcce4" }` ## Edit created subscription You can also edit the created subscription. ```http PATCH /objectEventSubscriptions/7dc53df5703e49b38670b1c468f47f1f ``` **Content-Type:** application/json **Request example:** ````text ``` json { "path": "myCustomPath", "destination": "aws-sns://arn:aws:sns:region:account-id:topicname" } ```` ```` **Response example:** ```text ``` json { "id": "7dc53df5703e49b38670b1c468f47f1f", "storageAccount": "dlbucketname", "storagePath": "data/sub=subtenantId/myfolder/mysubfolder", "destination": "aws-sns://arn:aws:sns:region:account-id:topicname", "eTag": 1, "subtenantId": "204a896c-a23a-11e9-a2a3-2a2ae2dbcce4" } ```` ```` ## Delete the subscription You can delete the created subscription if you do not want any further notifications. ```http DELETE /objectEventSubscriptions/7dc53df5703e49b38670b1c468f47f1f ```` **Response example:** ```http Status 204 No Content ``` ## Get the details of events for the given id You can get the details by ID of SNS topics to which you are already subscribed. ```http GET /objectEventSubscriptions/7dc53df5703e49b38670b1c468f47f1f ``` **Response Example:** ```json { "id": "7dc53df5703e49b38670b1c468f47f1f", "destination":"Endpoint=sb://iot-sbus-rc.servicebus.windows.net/;SharedAccessKeyName=idl-policy;SharedAccessKey=0LFL2eOdFJRCWDo6bDjtapEMq0XXX45dywFEChfhj4E=;EntityPath=idl-subscription-topic", "storageAccount": "datalake-rc-punazdl", "storagePath": "data/ten=punazdl/notification/nfr", "eTag": 2 } ``` ## Send event notification To enable Integrated Data Lake Service to send event notifications on provided SNS topic, the required permissions `SNS:GetTopicAttributes` and `SNS:Publish` should be granted. The following examples explain the functionality: 1. Allowing access specifically to Integrated Data Lake Service’s AWS account (Recommended approach): - Replace "\<`IDL AWS Account Id`>" with Integrated Data Lake Service’s AWS account id, which is `463822690055` - Replace "\<`SNS Topic ARN`>" with arn of the SNS topic subscribing to Integrated Data Lake Service ```json { "Version": "2008-10-17", "Id": "", "Statement": [ { "Sid": "AllowAccessToIDL", "Effect": "Allow", "Principal": { "AWS": "" }, "Action": [ "SNS:GetTopicAttributes", "SNS:Publish" ], "Resource": "" } ] } ``` 1. Allowing access specifically to all AWS accounts (Not Recommended as it allows every AWS account to access SNS topic) replace "\<`SNS Topic ARN`>" with arn of the SNS topic subscribing to Integrated Data Lake Service. ```json { "Version": "2008-10-17", "Id": "", "Statement": [ { "Sid": "AllowAccessToAll", "Effect": "Allow", "Principal": { "AWS": "*" }, "Action": [ "SNS:GetTopicAttributes", "SNS:Publish" ], "Resource": "" } ] } ``` # Delete Multiple Objects To delete multiple objects from the specified path, create an asynchronous jobs. Jobs which are completed with or without errors will be deleted periodically. Once the data is deleted, it cannot be recovered. Create asynchronous job to delete multiple objects for the specified paths. Jobs which are completed with or without errors would be deleted periodically. Once the data is deleted, it cannot be recovered. ## Provide delete permission The tenantAdmin can access this API and provide delete permission on folder or path, on the subtenants folder and on the root folder, except for the Time Series Import (TSI) folder. ```http POST /deleteObjectsJobs ``` **Content-Type:** application/json **Request Example:** ```json { "objects": [ { "path": "/IntegrationTest/IntegrationTestFile1" }, { "path": "/IntegrationTest/IntegrationTestFile2" } ] } ``` **Response Example:** ```json { "id": "63527e00-f4f6-4566-82bd-2f3ec81450fc", "status": "IN_PROGRESS", "progressDetails": { "totalObjects": 2, "inProgressObjects": 0, "deletedObjects": 0, "failedObjects": 0 } } ``` ## Get a list of deleted objects for the given ID ```http GET /deleteObjectsJobs/{id} ``` **Content-Type:** application/json **Response Example:** ```json { "id": "63527e00-f4f6-4566-82bd-2f3ec81450fc", "status": "COMPLETED", "progressDetails": { "totalObjects": 2, "inProgressObjects": 0, "deletedObjects": 2, "failedObjects": 0 } } ``` ## Get list of all deleted objects ```http GET /deleteObjectsJobs/ ``` **Content-Type:** application/json **Response Example:** ```json { "id": "63527e00-f4f6-4566-82bd-2f3ec81450fc", "status": "COMPLETED", "progressDetails": { "totalObjects": 2, "inProgressObjects": 0, "deletedObjects": 2, "failedObjects": 0 } }, { "id": "63527e00-f4f6-5677-82bd-2f3ec81458gb", "status": "COMPLETED", "progressDetails": { "totalObjects": 2, "inProgressObjects": 0, "deletedObjects": 2, "failedObjects": 0 } } ``` ## Get list of error details of the deleted objects ```http GET /deleteObjectsJobs/{id}/errorDetails ``` **Content-Type:** application/json **Response Example:** ```json { "id": "3b8aded7-d19c-4dff-af58-4a2840c3fe55", "subtenantId": "204a896c-a23a-11e9-a2a3-2a2ae2dbcce4", "progressDetails": { "totalObjects": 0, "inProgressObjects": 0, "deletedObjects": 0, "failedObjects": 0 }, "status": "COMPLETED", "objects": [ { "path": "string", "status": "FAILED" } ] } ``` # Downloading data from Integrated Data Lake This section describes how to download the data from Integrated Data Lake. ## Prerequisites The selection of methods solely depends on the requirement. You can download the data from Integrated Data Lake using below defined methods: 1. Generate signed URL for AWS / Aliyun or Shared Access Signatures (SAS) for Azure 1. Cross account access for AWS and Aliyun ### Generate Signed URL or Shared Access Signatures 1. To generate signed URL or Shared Access Signatures to download an object, follow below steps: - **Endpoint:** ```http POST /api/datalake/v3/generateDownloadObjectUrls ``` - **Content-Type:** application/json - **Request example:** ```json { "paths": [ { "path": "myfolder/mysubfolder/myobject.objext" } ] } ``` ```json { "paths": [ { "path": "myfolder/mysubfolder/myobject.objext" } ] } ``` ```json { "paths": [ { "path": "myfolder/mysubfolder/myobject.objext" } ] } ``` - **Response example:** ```json { "objectUrls":[ { "signedUrl":"https://datalake-integ-dide2-5234525690573.s3.eu-central-1.amazonaws.com/data/ten%3Ddide2/myfolder/mysubfolder/myobject.objext?X-Amz-Security-Token=Awervzdg23452xvbxd3434ddg&X-Amz-SignedHeaders=host&X-Amz-Expires=7200&X-Amz-Credentials=ASIATCES50453sdf&X-Amz-Signature=2e2342sfgsdfgsdgh", "path":"myfolder/mysubfolder/myobject.objext" } ] } ``` ```json { "objectUrls": [ { "Shared Access Signatures": "https://idltntprovisioningrc.blob.core.windows.net/datalake-rc-punrc118/data/ten=punrc118/folder1/mysensordata.log?sv=2018-11-09&spr=https&se=2020-03-06T12%3A59%3A23Z&sr=b&sp=r&sig=15ZV3ECHL0uQLFDZbQF%2FF5rPZrkZVsehRc%2FY9SmkZak%3D&2018-01-01T00%3A00%3A00.0000000Z", "path": "folder1/mysensordata.log" } ] } ``` ```json { "objectUrls": [ { "signedUrl": "https://datalake-integ-cdiot0-1627437476734.oss-cn-shanghai.aliyuncs.com/data/ten%3Dcdiot0/myfolder/mysubfolder/myobject.objext?Expires=1660906028&OSSAccessKeyId=LTAI5t9E24F6VDb6Q4hsyFNk&Signature=AfTxM4oLZZinmUU2jk8MHHT2Aks%3D&response-cache-control=No-cache", "path": "myfolder/mysubfolder/myobject.objext" } ] } ``` 1. You can use this signed URL to download one or multiple objects from the target folder. This URL is valid for 120 mins. Once the time limit is expired, you need to generate a new signed URL. ```text x-ms-blob-type: BlockBlob ``` **Endpoint:** ```http GET https://datalake-integ-dide2-5234525690573.s3.eu-central-1.amazonaws.com/data/ten%3Ddide2/myfolder/mysubfolder/myobject.objext?X-Amz-Security-Token=Awervzdg23452xvbxd3434ddg&X-Amz-SignedHeaders=host&X-Amz-Expires=7200&X-Amz-Credentials=ASIATCES50453sdf&X-Amz-Signature=2e2342sfgsdfgsdgh ``` ```http GET https://idltntprovisioningrc.blob.core.windows.net/datalake-rc-punrc118/data/ten=punrc118/folder1/mysensordata.log?sv=2018-11-09&spr=https&se=2020-03-06T12%3A59%3A23Z&sr=b&sp=r&sig=15ZV3ECHL0uQLFDZbQF%2FF5rPZrkZVsehRc%2FY9SmkZak%3D&2018-01-01T00%3A00%3A00.0000000Z ``` **Response example:** ```http This is sample text in the file being uploaded. ``` ### Cross account access for AWS only This method is used if you need a continuous access to the desired folder for download. Consider an example where you have an AWS account in which any application resides and this application needs to continuously access IDL folder. In such scenarios, Cross Account Access is useful. To use this method, you can follow below steps: 1. To create cross account on which access needs to be provided. ```http POST /crossAccounts ``` ```http Content-Type: application/json ``` **Request example:** ```json { "name": "testCrossAccount", "accessorAccountId": "960568630345", "description": "Cross Account Access for Testing", "subtenantId": "204a896c-a23a-11e9-a2a3-2a2ae2dbcce4" } ``` **Response example:** ```json { "id": "20234sd34a23a-11e9-a2a3-2a2sdfw34ce4", "name": "testCrossAccount", "accessorAccountId": "960768132345", "description": "Cross Account Access for Testing", "timestamp": "2019-09-06T21:23:32.000Z", "subtenantId": "204a896c-a23a-11e9-a2a3-2a2ae2dbcce4", "eTag": 1 } ``` 1. Once the cross account is created, perform cross account accesses to provide the desired access on desired prefix. ```http POST /crossAccounts/20234sd34a23a-11e9-a2a3-2a2sdfw34ce4/accesses ``` ```http Content-Type: application/json ``` **Request example:** ```json { "description": "Access to read from mysubfolder", "path": "myfolder/mysubfolder", "permission": "READ" } ``` **Response example:** ```json { "id": "781c8b90-c7b6-4b1c-993c-b51a00b35be2", "description": "Access to read from mysubfolder", "storageAccount": "dlbucketname", "storagePath": "data/ten=tenantname/myfolder/mysubfolder", "path": "myfolder/mysubfolder", "permission": "READ", "status": "ENABLED", "timestamp": "2019-11-04T19:19:25.866Z", "eTag": 1 } ``` 1. Once the accesses is provided, user can download data through CLI or using AWS SDK to the desired prefix, with the relevant accesses. Follow the commands given below to download the files from S3 bucket: `$ aws s3 cp s3://tgsbucket/myobject.objext .`\ `download: s3://tgsbucket/myobject.objext to ./myobject.objext` # Metadata Management from Integrated Data Lake Note Integrated Data Lake Service API version 4.\*.\*is not available for Local Private Cloud. Metadata management is a crucial aspect of data management within any Data Lake application. It involves the systematic organization, storage, retrieval, and maintenance of metadata, which is essentially data about data. This includes information about the characteristics, origin, usage, and relationships of the actual data stored in the Data Lake. ## Metadata collection operations Metadata collection is the systematic process of collecting metadata keys that manage all the relevant metadata information associated with the specific dataset or system. Metadata collections are classified into following categories: - Global collection - Custom collection - Reserved collection ### Creating a Metadata collection This API can be used to create metadata collection. Note Global collections are available by default in the system and cannot be created. Create a collection using the following endpoint: ```http POST/metadataCollections ``` **Sample Request:** ```json { "metadataCollectionId": "document_review", "label": "Document Review", "description": "usage and purpose of custom Metadata Collection" } ``` Note The users cannot create a collection with name "global" or "system", as it is reserved for system use. **Sample Response:** ```json { "metadataCollectionId": "document_review", "label": "Document Review", "description": "usage and purpose of custom Metadata Collection", "status": "DRAFT", "etag": 1 } ``` ### Retrieving the list of Metadata collections This API can be used to get the list of all the metadata collections. Global metadata collection (though default) is listed only when a metadata key is added into it. Retrieve collection list using the below endpoint: ```http GET/metadataCollections ``` **Sample Response:** ```json { "metadataCollections": [ { "metadataCollectionId": "document_review", "label": "Document Review", "description": "usage and purpose of custom Metadata Collection", "status": "DRAFT", "etag": 1 } ], "page": { "size": 0, "totalElements": 0, "totalPages": 0, "number": 0 } } ``` ### Retrieving the Metadata collection details This API can be used to get the details of the collection for the given id. Note With this endpoint, only collection details are fetched and not the details related to the keys inside the collection. Retrieve collection details using the below endpoint: ```http GET/metadataCollections/{metadataCollectionId} ``` **Sample Response:** ```json { "metadataCollectionId": "document_review", "label": "Document Review", "description": "usage and purpose of custom Metadata Collection", "status": "DRAFT", "etag": 1 } ``` ### Updating the Metadata collection details This API can be used to update the details of the collection for the given id. Note It is only possible to update the collections created by the user. Update collection details using the below endpoint: ```http PATCH/metadataCollections/{metadataCollectionId} ``` **Sample Request:** ```json { "label": "Document Review", "description": "usage and purpose of custom Metadata Collection", "status": "DRAFT" } ``` Note Collection status can be updated only from DRAFT to PUBLISHED. **Sample Response:** ```json { "metadataCollectionId": "document_review", "label": "Document Review", "description": "usage and purpose of custom Metadata Collection", "status": "DRAFT", "etag": 1 } ``` ### Deleting the Collection This API can be used to delete the collection of the given id. Delete the collection using the following endpoint: ```http DELETE/metadataCollections/{metadataCollectionId} ``` Note - It is not possible to delete Global collection. - It is only possible to delete the custom collection in draft status. ## Metadata Keys Operations Metadata keys are the unique identifiers used to represent specific attributes or characteristics associated with the data in a key-value pair structure. ### Creating a Metadata Key This API can be used to create metadata key. This API allows admin users to add metadata keys to metadata collection. By default, admin users can add keys directly to Global metadata collection. There are various configuration parameters available while defining a metadata key. These configuration parameters helps admin users to define keys to accept specific metadata values. The configuration parameters are defined below: - Value Type: This is the data type that defines the type of metadata value which are accepted for this key. The available value type details are described below: | Value Type | Use Case | | ---------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Enum | Metadata values are pre-defined and user can select one of the value. The available values are seen while defining a key, which supports maximum of 300 values. The special characters supported in Enum values are - Alphanumeric characters (A-Z, a-z, 0-9) - Hyphen ("-") - Forward slash ("/") - Parentheses ("(" and ")") - Period (".") - Colon (":") - Comma (",") - Underscore ("\_") - spaces(" ") but not leading and trailing | | EnumList | Metadata values are pre-defined and user can select one or more values. The available values are seen while defining a key, which supports maximum of 300 values. Additionally user can select maximum and minimum values. The special characters supported in EnumList values are - Alphanumeric characters (A-Z, a-z, 0-9) - Hyphen ("-") - Forward slash ("/") - Parentheses ("(" and ")") - Period (".") - Colon (":") - Comma (",") - Underscore ("\_") but not leading or trailing - spaces(" ") but not leading and trailing | | String | User has to input one metadata value as string. Additionally, maximum and minimum length are also configured. | | StringList | User has to input one or more metadata values as string. Additionally, maximum and minimum number of accepted values are also configured. | | Long Text | User has to input long values of string e.g. "notes or information as metadata for data lake resources". Long Text support maximum of 2048 characters | - Field name and Field values: These are the additional details that admin users can use to put constraints on metadata values as described below: | Value Type | Field Name | Field Value | | ---------- | ---------- | ----------------------------------------------------------------------------------------------------------------------------------------------- | | Enum | options | List of comma separated acceptable values for Enum in capital letters | | EnumList | options | List of comma separated acceptable values for Enum in capital letters | | | minSize | Minimum number of values that can be selected from the enum list. It has to be greater than 0 and less than number of options. | | | maxSize | Maximum number of values that can be selected from the enum list. It has to be greater than minSize and less than or equal to number of options | | String | options | List of comma separated acceptable values for Enum in capital letters | | | minLength | Minimum length of character set in the list that can be accepted as value. It has to be greater than 0. | | | maxLength | Maximum length of character set in the list that can be accepted as value. It has to be greater than or equal to minLength | | StringList | options | List of comma separated acceptable values for Enum in capital letters | | | minSize | Minimum number of values that can be selected from the enum list. It has to be greater than 0 and less than number of options. | | | maxSize | Maximum number of values that can be selected from the enum list. It has to be greater than minSize and less than or equal to number of options | - applyOn: This parameter is used to define where it should be applied i.e., objects (files) or folders based on various requirements. | Options | Use Case | | ------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | OBJECTS_ONLY | Can be used for any metadata that is applicable only at object level | | FOLDERS_ONLY | Can be used to set the metadata in the entire hierarchy which should not be changed at any level e.g. Business Sensitivity or Classification Levels. To be avoided for the metadata which is updated regularly | | FOLDERS_AND_OBJECTS | Can be used where metadata is applicable on entire hierarchy but can be changed if required | - Rule Key: This indicated if this key can be used to create rules. Note - Minimum value for key should be 2 characters. - The key consists of alphabetic characters or a combination of alphabetic and numeric characters. It should not be composed of numerical digits. Recommendations: - It is advised to keep the key in DRAFT status when created. Many details of the key cannot be updated once key is PUBLISHED. Create a collection using the following endpoint: ```http POST/metadataCollections/{metadataCollectionId}/keys ``` **Sample Request:** ```json { "key": "country_of_origin", "label": "Country of Origin", "description": "description of metadata key and its usage", "valueType": "String", "additionalDetails": [ { "fieldName": "StringType", "fieldValues": [ "ALPHABETIC" ] } ], "applyOn": "OBJECTS_ONLY", "isMandatory": true, "isRuleKey": false, "isSearchable": false, "status": "DRAFT" } ``` **Sample Response:** ```json { "key": "country_of_origin", "label": "Country of Origin", "description": "description of metadata key and its usage", "valueType": "String", "additionalDetails": [ { "fieldName": "StringType", "fieldValues": [ "ALPHABETIC" ] } ], "applyOn": "OBJECTS_ONLY", "isMandatory": true, "isRuleKey": false, "isSearchable": false, "status": "DRAFT", "etag": 1 } ``` ### Deprecating Enum value This API allows admin and standard user to deprecate Enum value from the configured Enum datatype. The conditions to deprecate Enum value are defined below: - All the values from the configured Enum key cannot be deprecated. There should be at least one value present in the Enum. - User cannot deprecate the value which is not present in the ENUM options. - Deprecated Enum values are not available to apply metadata. - User can restore the deprecated value in the same request by providing the value in "options". Restore activity is opposite of deprecation. - All the values marked for deprecation are restored in a single request. Note - If any Enum value is deprecated, there is no impact on any functional activity associated with that Enum value. These activities include metadata changes and ABAC evaluation. - If metadata is updated on any object containing deprecated Enum value, user should change the value to other option. **Sample Enum value deprecation:** ```json { "label": "Business Sensitivity", "description": "Description of metadata key and its usage", "additionalDetails": [ { "fieldName": "DeprecatedOptions", "fieldValues": [ "deprecated_value_1", "deprecated_value_2" ] }, { "fieldName": "Options", "fieldValues": [ "value_1", "value_2" ] } ] } ``` ### Retrieving the list of Metadata keys This API can be used to get the list of all the metadata keys in the provided metadata collection. Retrieve list of keys using the below endpoint: ```http GET /metadataCollections/{metadataCollectionId}/keys ``` **Sample Response:** ```json { "metadataKeys": [ { "key": "country_of_origin", "label": "Country of Origin", "description": "description of metadata key and its usage", "valueType": "String", "additionalDetails": [ { "fieldName": "StringType", "fieldValues": [ "ALPHABETIC" ] } ], "applyOn": "OBJECTS_ONLY", "isMandatory": true, "isRuleKey": false, "isSearchable": false, "status": "DRAFT", "etag": 1 } ], "page": { "size": 0, "totalElements": 0, "totalPages": 0, "number": 0 } } ``` ### Retrieving the key details This API can be used to get the details of the defined key in the collection. Retrieve key details using the below endpoint: ```http GET /metadataCollections/{metadataCollectionId}/keys/{metadataKey} ``` **Sample Response:** ```json { "key": "country_of_origin", "label": "Country of Origin", "description": "description of metadata key and its usage", "valueType": "String", "additionalDetails": [ { "fieldName": "StringType", "fieldValues": [ "ALPHABETIC" ] } ], "applyOn": "OBJECTS_ONLY", "isMandatory": true, "isRuleKey": false, "isSearchable": false, "status": "DRAFT", "etag": 1 } ``` ### Updating the Metadata key details This API allows the admin users to update metadata key configuration. Update details for metadata key depends on the status of the key. The following parameters can be updated based on key status: | Parameter | Draft Status | Published Status | | ------------------- | ----------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | label & description | Can be updated | Can be updated | | Additional Details | Can modify or remove existing constraints, or add new constraints | Can change existing fieldValues to extend the already defined range e.g. minValue can be reduced or maxValue can be increased or new Enum value can be added. Cannot remove any existing constraints | | isMandatory | Can be updated to true or false | Can be updated to true or false | | isRuleKey | Can be updated to true or false | Can be updated only if metadata collection in which this ruleKey is used is in DRAFT status | | applyOn | Can be updated to any of the available Enum values | Cannot be updated | | isSearchable | Can be updated to true or false | Cannot be updated | Update key details using the below endpoint: ```http PATCH /metadataCollections/{metadataCollectionId}/keys/{metadataKey} ``` **Sample Request:** ```json { "label": "Business Sensitivity", "description": "description of metadata key and its usage", "additionalDetails": [ { "fieldName": "StringType", "fieldValues": [ "ALPHABETIC" ] } ], "isMandatory": true, "isRuleKey": false, "status": "DRAFT" } ``` **Sample Response:** ```json { "key": "country_of_origin", "label": "Country of Origin", "description": "description of metadata key and its usage", "valueType": "String", "additionalDetails": [ { "fieldName": "StringType", "fieldValues": [ "ALPHABETIC" ] } ], "applyOn": "OBJECTS_ONLY", "isMandatory": true, "isRuleKey": false, "isSearchable": false, "status": "DRAFT", "etag": 1 } ``` ### Deleting the Keys This API allows tenant admin to delete metadata key under the provided metadata collection provided in the request. Note It is only possible to delete the key, which is in Draft status. Delete the collection using the following endpoint: ```http DELETE /metadataCollections/{metadataCollectionId}/keys/{metadataKey} ``` ## Metadata Rules Operations Metadata Rules are a set of predefined conditions that determines the metadata tied to a custom collection that is applicable to Integrated Data Lake resources. ### Creating a Metadata Rule This API creates metadata rules for metadata collection corresponding to a metadata key. Note - Status of the rule is governed by collection status - Rule can be created for collection in DRAFT as well as PUBLISHED status - If collection is in DRAFT status, rule is not applied and collection attributes will not be available to enter metadata values - If collection is in PUBLISHED status, rule gets applied immediately Recommendations: - It is advised to Keep the collection in DRAFT status while creating rule. Do not publish collection unless certain about rule key and its value used in the rule. Create a metadata rule using the following endpoint: ```http POST /metadataCollections/{metadataCollectionId}/metadataRules ``` **Sample Request:** ```json { "name": "Document Review Rule", "key": "business_sensitivity", "value": "STRICTLY_PRIVATE" } ``` **Sample Response:** ```json { "id": "0860f696-af41-4d8d-a104-c1dd508de97a", "name": "Document Review Rule", "key": "business_sensitivity", "value": "STRICTLY_PRIVATE", "etag": 1 } ``` ### Retrieving the list of Metadata rules This API lists all the metadata rules associated with a collection. Retrieve the list of all the metadata rules using the following endpoint: ```http GET /metadataCollections/{metadataCollectionId}/metadataRules ``` **Sample Response:** ```json { "metadataRules": [ { "id": "0860f696-af41-4d8d-a104-c1dd508de97a", "name": "Document Review Rule", "key": "business_sensitivity", "value": "STRICTLY_PRIVATE", "etag": 1 } ], "page": { "size": 0, "totalElements": 0, "totalPages": 0, "number": 0 } } ``` ### Retrieving the metadata rule details This API provides the metadata rule details for the given id. Retrieve the metadata rule details using the following endpoint: ```http GET /metadataCollections/{metadataCollectionId}/metadataRules/{id} ``` **Sample Response:** ```json { "id": "0860f696-af41-4d8d-a104-c1dd508de97a", "name": "Document Review Rule", "key": "business_sensitivity", "value": "STRICTLY_PRIVATE", "etag": 1 } ``` ### Updating the metadata rules This API updates the metadata rule details for the given id. Rule can be updated only in DRAFT status. Update the metadata rule details using the below endpoint: ```http PATCH /metadataCollections/{metadataCollectionId}/metadataRules/{id} ``` **Sample Request:** ```json { "name": "Document Review Rule", "key": "business_sensitivity", "value": "STRICTLY_PRIVATE" } ``` **Sample Response:** ```json { "id": "0860f696-af41-4d8d-a104-c1dd508de97a", "name": "Document Review Rule", "key": "business_sensitivity", "value": "STRICTLY_PRIVATE", "etag": 1 } ``` ### Deleting the metadata rule This API deletes the metadata rule provided in the request. Note It is only possible to delete the rule, which is in Draft status. Delete the collection using the following endpoint: ```http DELETE /metadataCollections/{metadataCollectionId}/metadataRules/{id} ``` ## Attribute Based Access Control Attribute Based Access Control (ABAC) is an authorization model that evaluates attributes to determine access. For more information on ABAC, refer to [Attribute Based Access Control configuration](../core-resourceaccessmanagement/ABAC-based-policy.html). The Attribute Based Access Control for Integrated Data Lake operations are as follows: - **Upload an Object via App Gateway with Metadata** This PUT method is used to upload an object in the path is permitted as per the policy created for the user by Tenant Admin. - Access to upload a file is evaluated at the prefix (parent folder) level for the user. - For evaluating the access control, refer to [Object and folder operation](api-integrated-data-lake-object-and-folder-operation.html). Endpoint: ```http PUT /api/datalake/v4/objects/{path} ``` - **Download Object via App Gateway** This GET method is used to download an object in the path is permitted as per the policy created for the user by Tenant Admin. - Access to download a file is evaluated at the object or file level for the user. Endpoint: ```http GET /objects/{path} ``` - **Create empty folder with Metadata** This POST method is used to create a folder in the path is permitted as per the policy created for the user by Tenant Admin. - Access to create a folder is evaluated at the prefix (parent folder) level for the user. - For evaluating the access control, refer to [Object and folder operation](api-integrated-data-lake-object-and-folder-operation.html). Endpoint: ```http POST /folders ``` - **Delete an empty Folder** This DELETE method is used to delete a folder in the path is permitted as per the policy created for the user by Tenant Admin. - Access to delete a folder is evaluated at the prefix (folder) level for the user. - If permitted, metadata associated with the folder are deleted. Endpoint: ```http DELETE /folders/{path} ``` - **Delete Object** This DELETE method is used to delete an object on the path which is permitted as per the policy created for the user by Tenant Admin. - Access to delete file is evaluated at the object (file) level for the user. - If permitted, metadata associated with the folder are deleted. Endpoint: ```http DELETE/{path} ``` - **List Objects** This GET method is used to access to list objects at a given path for the tenant is evaluated as per the policy created by the Tenant Admin. - User can see the folder hierarchy, if access on the parent folder is provided with propagation depth as -1. For more information on propagation depth, refer to [Object and folder operation](api-integrated-data-lake-object-and-folder-operation.html). - If user does not have access to any resources, list response is empty. List response will not give any error. Endpoint: ```http GET /listObjects ``` - **Add Metadata** Users with "WRITE" action can add or update the metadata on the resources. Endpoint: ```http /objectMetadata/{path} ``` - **Get Metadata** Users with "LIST" access can see UI or fetch API the metadata on the resources. Endpoint: ```http /objectMetadata/{path} ``` Note In Azure Cognitive Search, a single search query can include up to 1000 clauses. This restriction applies to the entire search expression, including any filters used. If your application generates complex queries programmatically, it's advisable to design them to stay within this limit to avoid potential issues. ## Access Control Actions The access control actions for Integrated Data Lake operations are described below: | Operation | List | Read + List | Create + List | Delete + List | Create + Read + List | Delete + Read + List | Delete + Create + Read + List | | ---------------------------------------------------------------------- | ---- | ----------- | ------------- | ------------- | -------------------- | -------------------- | ----------------------------- | | View resources (files and folders) and System metadata | | | | | | | | | View Metadata (user defined metadata) | | | | | | | | | Upload file with metadata | | | | | | | | | Download the files | | | | | | | | | Create folder with metadata | | | | | | | | | Update Metadata (independent operation) | | | | | | | | | Delete file(s) or empty folder | | | | | | | | | Search objects - File name + Metadata (system + user defined metadata) | | | | | | | | ## Action Dependency Table The dependencies for Integrated Data Lake actions are described below: | Action | Description | | ------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------- | | List Resources `mdsp:core:idl:prefix:list` | Listing of resources within the storage. | | Download Files `mdsp:core:idl:prefix:read` | Downloading files from the storage. This depends on the list of the resources. | | Upload files and create folders `mdsp:core:idl:prefix:write` | Uploading the files and creating empty folder with the storage. This depends on the list of the resources and downloading files from the storage. | | Delete files and folder `mdsp:core:idl:prefix:delete` | Deleting the files and folder from the storage. This depends on the list of the resources and downloading files from the storage. | # Metadata and Object search The Metadata and Object search can be used to search and analyze big volumes of objects quickly via a highly scalable text search. The search accepts filter parameters such as object name, location, created date, size and tags. Filters can be combined using "and" and "or" conditions. The search results can be sorted by any one of the filter parameters and limited to maximum number of records per result set. If there are more results available than the defined maximum, a `searchNext` property is returned in the response. This property specifies the search request, which returns the remaining the results. It is included in every subsequent response, until the end of the list. ## Supported Operators The search filter supports the following operators for different data types as shown in the table: | Data Type | Operators | Description | | ------------ | ---------- | --------------------------------------------------------------------------------------- | | String | contains | - Searches for generic strings - Uses wildcard to search strings may impact performance | | | eq | - Direct match and recommends for quick search - Wildcard characters are not allowed | | | startswith | - Matches the initial characters of the string - Uses a wildcard character at the end | | Date | before | Searches before the specified date | | | between | Searches between the search dates | | | after | Searches after the specified date | | Integer | eq | Matches exactly | | | gt | Searches for integers greater than the specified value | | | lt | Searches for integers less than the specified value | | Array (tags) | in | Matches the string within a list of strings | ## Get a list of metadata for an object - 'None' Operator **Request URL:** ```http {{gatewayUrl}}/api/datalake/v3/objectMetadata?filter=%7B%0A%20%20%22none%22%3A%7B%0A%20%20%22name%22%3A%20%7B%0A%20%20%20%20%22contains%22%3A%20%22sensor%22%0A%20%20%20%20%7D%0A%20%20%7D%0A%7D ``` Note - **Filter Request**: `filter={ "none":{ "name": { "contains": "sensor" } }` - Filter request is URL encoded **Content-Type**: application/json **Response Example:** ```json { "objectMetadata": [ { "name": "test1sensor.csv", "location": "Folder/test1sensor.csv", "size": 15, "created": "2018-10-01T09:21:36.559Z", "updated": "2018-10-03T09:21:36.559Z", "tags": [ "tag1", "tag3" ], "metadataURL": "https://gateway.eu1-int.mindsphere.io/api/v3/dlstorage/objectMetadata/Folder1/test1sensor.csv" } ] } ``` ## Get a list of metadata for an object - 'OR' Operator **Request URL:** ```http {{gatewayUrl}}/api/datalake/v3/objectMetadata?filter=%7B%0A%20%20%22or%22%3A%20%20%7B%0A%20%20%20%20%22name%22%3A%20%7B%20%22contains%22%3A%20%22sensor%22%7D%2C%0A%20%20%20%20%22created%22%3A%20%7B%22after%22%3A%20%222018-10-24T11%3A42%3A53.983Z%22%7D%2C%0A%20%20%20%20%22size%22%3A%20%7B%20%22eq%22%3A37%7D%2C%0A%20%20%20%20%22tags%22%3A%7B%20%22in%22%3A%5B%22tag1%22%2C%22tag3%22%5D%7D%0A%20%20%7D%0A%7D&maxRecords=1 ``` Note - **Filter Request**: `filter={ "or": { "name": { "contains": "sensor"}, "created": {"after": "2018-10-24T11:42:53.983Z"}, "size": { "eq":37}, "tags":{ "in":["tag1","tag3"]} } }` - Filter request is URL encoded **Content-Type:** application/json **Response Example:** ```json { "objectMetadata": [ { "name": "test1sensor.csv", "location": "Folder/test1sensor.csv", "size": 15, "created": "2018-10-01T09:21:36.559Z", "updated": "2018-10-03T09:21:36.559Z", "tags": [ "tag1", "tag3" ], "metadataURL": "https://gateway.eu1-int.mindsphere.io/api/v3/dlstorage/objectMetadata/Folder1/test1sensor.csv" } ], "sort": [ { "field": "name", "order": "ASC" } ], "searchnext": [ { "field": "name", "next": "test2sensor" }, { "field": "_id", "next": "2222222" } ] } ``` ## Get a list of metadata for an object - 'AND' Operator **Request URL:** ```http {{gatewayUrl}}/api/datalake/v3/objectMetadata?filter=%7B%0A%20%20%20%20%22and%22%3A%20%20%7B%0A%20%20%20%20%22name%22%3A%20%7B%20%22contains%22%3A%20%22sensor%22%7D%2C%0A%20%20%20%20%22created%22%3A%20%7B%22after%22%3A%20%222018-10-24T11%3A42%3A53.983Z%22%7D%2C%0A%20%20%20%20%22size%22%3A%20%7B%20%22eq%22%3A37%7D%2C%0A%20%20%20%20%22tags%22%3A%7B%20%22in%22%3A%5B%22tag1%22%2C%22tag3%22%5D%7D%0A%20%20%7D%0A%7D ``` Note - **Filter Request**: `filter={ "and": { "name": { "contains": "sensor"}, "created": {"after": "2018-10-24T11:42:53.983Z"}, "size": { "eq":37}, "tags":{ "in":["tag1","tag3"]} } }` - Filter request is URL encoded **Content-Type:** application/json **Response Example:** ```json { "objectMetadata": [ { "name": "test1sensor.csv", "location": "Folder/test1sensor.csv", "size": 15, "created": "2018-10-01T09:21:36.559Z", "updated": "2018-10-03T09:21:36.559Z", "tags": [ "tag1", "tag3" ], "metadataURL": "https://gateway.eu1-int.mindsphere.io/api/v3/dlstorage/objectMetadata/Folder1/test1sensor.csv" } ] } ``` # Metadata Search Note This section is available only for Virtual Private cloud (VPC). The Metadata search can be used to search and analyze big volumes of objects quickly via a highly scalable text search. The search accepts request parameters such as System metadata (e.g. object name, location, created date, size) and Custom metadata (e.g. metadata keys). Filters can be combined using multiple comparative and relational operators based on the the datatype of the metadata. The search result will be available as collection of files and folders which match the provided search criteria for the provided storage path for which user has access. Search API honors the ABAC policies as applicable to the user. If there are more results available than the defined maximum i.e. page size, a `nextToken` property is returned in the response. This property specifies the search request, which returns the remaining the results. It is included in every subsequent response, until the end of the list. ## Supported Operators The metadata types and their supported search operators are displayed in the table: | Metadata Type | Parameter Name (Datatype) | Operator | Description | | --------------- | ------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | System metadata | size (INT) | EQ → Equal GT → Greater than GE → Greater than or equal NE → Not equal LT → Less than LE → Less than or equal | System metadata includes parameters like object name, size, type, and last updated date | | | fileName (STRING) | EQ → Equal NE → Not equal startsWith → starts with contains → contains set of characters within file name | Name of the object | | | key (STRING) | EQ → Equal NE → Not equal startsWith → starts with contains → contains set of characters within file name | path of the object | | | documentType (STRING) | EQ → Equal NE → Not equal | Type of the document that is file or folder. Supported values are `OBJECT` and `FOLDER` | | | lastModified (DATE) | EQ → Equal NE → Not equal GT → Greater than LT → Less than GE → Greater than or equal LE → Less than or equal after → after the provided input date before → before the provided input date | Last updated date of the object (file) | | Custom metadata | ENUMLIST ENUM STRING STRINGLIST | EQ → Equal NE → Not equal “==” → Equal “!=” → Not equal IN → Value within ENUMLIST NOT IN → Value NOT IN ENUMLIST | Support List of predefined values **Note:** Ensure that the search criteria should always be in sequence - `Value IN ENUMLIST` List should always be on the right hand side of the operator for IN and NOT IN Support ONE predefined values from the list Any character stream List of character streams **Note:** Ensure that the search criteria should always be in sequence - `Value IN STRINGLIST` List should always be on the right hand side of the operator for IN and NOT IN | ## Search API Note Search operation honors Attribute Based Access Control (ABAC). User will see only those objects in search result for which he as access. **Endpoint:** `POST /search` **Request:** ```json { "currentPath": "20MayFolder/", "systemMetadataCriteria": "system.documentType eq 'OBJECT' or system.documentType eq 'FOLDER'", "criteriaJoinOperator": "and", "customMetadataCriteria": "global.file_status eq 'PUBLIC'", "nextToken": "eyJhdXRob3JpemVkUGFnZSI6Mywic2tpcCI6NCwibWF4U2l6ZSI6NH0=", "maxSize": 200 } ``` ### Request Parameters Description The overview of the parameters required for making data requests, detailing search paths are displayed in the table: | Parameter | Datatype | Description | Recommended Usage | | ---------------------- | -------- | ---------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | currentPath | STRING | Specifies the directory or folder path where search needs to be performed | This will always be folder. Search will be performed in the entire hierarchy of this path. The result would be presented as a flat hierarchy listing all the files/folders in the path **Note:** `currentPath` cannot be a filename. It could possibly be root path ('/') | | systemMetadataCriteria | STRING | System metadata criteria supports 5 search parameters - size - fileName - path - resourceType lastModified | Check [Supported operators](api-integrated-data-lake-samples-metadata-search.html#supported-operators) to create the criteria related to system metadata System metadata criteria has a limit of 3000 characters **Sample:** `system.fileName eq 'test.txt' and system.documentType eq 'OBJECT'` | | criteriaJoinOperator | STRING | Condition operator to join the system and custom metadata criteria | Possible value are `AND` and `OR` only | | customMetadataCriteria | STRING | Custom metadata support four datatypes - ENUM, ENUMLIST, STRING, STRINGLIST | Check [Supported operators](api-integrated-data-lake-samples-metadata-search.html#supported-operators) to create the criteria related to custom metadata Custom metadata criteria has a limit of 10,000 characters **Sample:** `global.accessCriteria eq 'PUBLIC'` | | nextToken | STRING | Value of next token to fetch next request results. Combination of access control pages and end response | | | maxSize | INTEGER | Maximum number of objects in a single request | Maximum number of objects that can be fetched in single request is 1,000 | **Response:** ```json { "storageAccount": "datalake-dev-aaab", "storagePath": "data/ten=tenantname/{path} OR data/sub=subtenantId{path}", "objects": { "files": [ { "key": "data/ten=dide2/commands.txt", "lastModified": "2018-10-03T09:21:36.559Z", "contentSize": 1000, "storageClass": "HOT" } ], "folders": [ { "key": "data/ten=dide2/e2eTest/" } ] }, "page": { "size": 1000, "nextToken": "1BpNN1QT3bG+k2dJG1TgmquqCzzCjbiuYymgjL91UChw//OBKBgBrmUcZpF6fT6C428OjxaMntOk+doIH45VMRAWt40WHl58ddeTuzsMCsd9T/nMjddi2SQ==" }, "isTruncated": true ``` # Working with Metadata You can create or update metadata. A custom metadata can be added while uploading the object. Once the object is uploaded, default system metadata will be attached. If environment is operating for a subtenant object, the subtenant should be specified in the request query parameter. A maximum of 8 tags are allowed, where each tag can have a maximum of 128 characters. ## Assign Tags to an object ```http PUT /objectMetadata/myfolder/mysubfolder/myobject.objext ``` ```http PUT /objectMetadata/folder1/mysensordata.log ``` ```http PUT /objectMetadata/myfolder/mysubfolder/myobject.objext ``` **Content-Type**: application/json **Request example:** ```json { "tags": [ "tag1", "tag2", "tag3" ] } ``` ```json { "tags": [ "tag1", "tag2", "tag3" ] } ``` ```json { "tags": [ "tag1", "tag2", "tag3", "tag4", "tag5" ] } ``` **Response example:** ```http Status 201 Created ``` ## Get Metadata for a single object ```http GET /objectMetadata/myfolder/mysubfolder/myobject.objext ``` ```http GET /objectMetadata/folder1/mysensordata.log ``` ```http GET /objectMetadata/myfolder/mysubfolder/myobject.objext ``` **Content-Type:** application/json **Response example:** ```json { "name": "myobject.objext", "location": "myfolder/mysubfolder/myobject.objext", "size": 25, "lastModified": "2018-10-03T09:21:36.559Z", "tags": [ "tag1", "tag2", "tag3" ], "subtenantId": "204a896c-a23a-11e9-a2a3-2a2ae2dbcce4" } ``` ```json { "name": "mysensordata.log", "location": "folder1/mysensordata.log", "size": 14, "tags": [ "tag1", "tag2", "tag3", "tag4", "tag5", "tag6", "tag7", "tag8" ], "lastModified": "2020-03-06T10:59:21Z" } ``` ```json { "name": "myobject.objext", "location": "myfolder/mysubfolder/myobject.objext", "size": 25, "lastModified": "2021-10-03T09:21:36.559Z", "tags": [ "tag1", "tag2", "tag3", "tag4", "tag5" ], "subtenantId": "204a896c-a23a-11e9-a2a3-2a2ae2dbcce4" } ``` \*\[update metadata\]: In IDL, metadata is available in the form of values and can be provided in folders or objects. Metadata values provides a way to categorize content, focus on key concepts or themes. # Time Series Import ## Initiate a new bulk import job This POST method is used to initiate a new bulk import job with the provided parameters. The parameters that you can provide in the body of the request and are a set of entity IDs, one or more property sets, and a time range. All the data in the time range for all the property sets for all the entities will be imported to data lake. ```http POST /timeSeriesImportJobs ``` **Request Example:** ```json { "name": "testazure26-11", "destination": "testazure26-11", "aspectNames": [ "idl_aspect1" ], "assetIds": [ "222f32d5d089408bbaaa48cecd613b4b" ], "from": "2018-10-24T19:29:58.000Z", "to": "2018-11-24T19:29:58.000Z" } ``` **Response Example:** ```json { "id": "c4a4532a28c44b9aa5f2c31fd8d6da8b", "name": "testazure26-14", "status": "PENDING", "destinationPath": "testazure26-14" } ``` ## Get a list of time series bulk import jobs This GET method is used to get the status of all current time series bulk import jobs initiated by environment. Periodically, successfully completed job records gets deleted and they will not be part of result set for this API. ```http GET /timeSeriesImportJobs ``` **Response Example:** ```json { "timeSeriesImportJobs": [ { "id": "c71b8929e52c4b1c8959c33d012ed8d6", "name": "IntegTest671b007f-16bb-47b6-83b1-5103dcac2012", "status": "FAILED", "destinationPath": "IntegTest/" } ], "page": { "size": 1, "totalElements": 38, "totalPages": 38, "number": 0 } } ``` ## Get a list of time series bulk import jobs for given ID This GET method is used to get the current status of a time series bulk import job by Id. Periodically, successfully completed job records gets deleted and they will not be part of result set for this API. ```http GET /timeSeriesImportJobs/c71b8929e52c4b1c8959c33d012ed8d6 ``` **Response Example:** ```json { "id": "c71b8929e52c4b1c8959c33d012ed8d6", "name": "IntegTest671b007f-16bb-47b6-83b1-5103dcac2012", "status": "FAILED", "destinationPath": "IntegTest/" } ``` ## Get a details of time series bulk import jobs for given ID ```http GET /timeSeriesImportJobs/c71b8929e52c4b1c8959c33d012ed8d6/details ``` **Response Example:** ```json { "id": "1b67dba3a7f0477382c5a8f9ad50a46d", "from": "2018-10-24T19:29:58.000Z", "to": "2018-10-24T19:29:58.000Z", "status": "SUCCESS", "progress": 100.0, "aspectNames": [ "idl_aspect1" ], "assetIds": [ "222f32d5d089408bbaaa48cecd613b4b" ], "destinationPath": "testazure24-13/", "fileCount": 0, "name": "testazure24-13" } ``` ## Delete the time series bulk import jobs ```http Delete /timeSeriesImportJobs/c71b8929e52c4b1c8959c33d012ed8d6/details ``` **Response Example:** ```json Status 204 No Content ``` # Toggle Searchable Note This section is available only for Virtual Private cloud(VPC). When metadata is defined, metadata key can be configured as `searchable` or `nonsearchable`. User can now toggle this configuration on metadata keys even after the key is in PUBLISHED state. To facilitate these configuration adjustments across existing data, a dedicated POST endpoint (/api/datalake/v4/retrofit-jobs), is utilized. This endpoint processes a list of specified retrofit operations, subsequently initiating an asynchronous job to apply the requisite changes to relevant folders and objects. The supported operations include transitioning a metadata key's configuration from non-searchable to searchable and conversely, from searchable to non-searchable. ## Create Update Metadata Definition API The endpoint is used to update the searchability status of a published metadata key. It supports transitioning a key from 'Searchable' to 'Non-searchable' and vice versa. Upon request, an asynchronous background job is initiated to perform a 'retrofit', ensuring that all folders and files associated with the updated metadata key are also modified accordingly. This is an asynchronous operation and will take some time to complete. Updating metadata definition is heavy operation and should be used cautiously, especially on high data volume. Note - There might be some interference with existing operation when update metadata definition operation is IN_PROGRESS state - It is recommended not to initiate multiple update metadata definition operation simultaneously. **Endpoint:** `POST /metadataDefinitionUpdateJobs` **Request:** ```json { "operations": [ { "key": "country_of_origin", "metadataCollectionId": "document_review", "operationType": "SEARCHABLE_TO_NONSEARCHABLE", "usedInSDSPolicy": false, "additionalDetails": [ { "fieldName": "StringType", "fieldValues": [ "ALPHABETIC" ] } ] } ] ``` ### Request Parameter Description The overview of the parameters, datatypes and detailing search paths are displayed in the table: | Parameter | Datatype | Description | Recommended Usage | | ----------------- | -------- | ------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | operationType | STRING | Type of metadata definition update operation to be performed | Supported operations are: - **SEARCHABLE_TO_NONSEARCHABLE (case insensitive):** The system first validates for any active SDS policies associated with the metadata key. For the operation to proceed, the request must explicitly include the usedInSDSPolicy parameter set to false. Upon successful validation, the metadata key definition is updated (for example, from 'Searchable' to 'Non-searchable'). Following this, an asynchronous background job is initiated to perform a 'retrofit' which updates all associated folders and objects. - **NONSEARCHABLE_TO_SEARCHABLE (case insensitive):** Update the metadata key definition from non-searchable to searchable and retrofit the associated folders and objects where the key is used. **Note:** Multiple operations can be performed in a single request. Ensure that these operations are performed on different metadata key. | | usedInSDSPolicy | BOOLEAN | Any active policies defined on this metadata key as part of attribute based access control (ABAC) | Possible values are: - **False:** This metadata key is not used in any access control policies. System will proceed with the operation and initiate the configuration change - **Null or True:** This metadata key is used in any access control policies. System will give ERROR! User will have to ensure that there are no active policies in the system using this metadata key **Caution:** Admin user is responsible to provide this information correctly, especially for Searchable to Non-Searchable. If there are policies defined using this metadata key, it would cause metadata integrity issue if this operation is invoked. This metadata key would no longer be available for policy evaluation. System will not validate this information. | | additionalDetails | - | Optional information related to metadata key and its associated values | This information is for future use when additional operations will be added to operationType. This filed is not required for existing operations | **Response:** ```json { "jobId": "0b68f101-4016-42cc-b511-05774f2f47a1", "status": "IN_PROGRESS", "created": "2025-07-28T14:53:26Z", "path": "data/ten=qvstnt3/", "operationDetailsList": [ { "key": "country_of_origin", "metadataCollectionId": "document_review", "operationType": "SEARCHABLE_TO_NONSEARCHABLE", "usedInSDSPolicy": false, "additionalDetails": [ { "fieldName": "StringType", "fieldValues": [ "ALPHABETIC" ] } ] } ] } ``` ## Request Parameter Details The overview of the parameters, datatypes and detailing paths are displayed in the table: | Parameter | Datatype | Description | Recommended Usage | | -------------------- | -------- | ------------------------------------------------------------- | ----------------------------------------------------------------- | | jobId | STRING | Identifier of the asynchronous job created for this operation | User can track the status of this operation using this identifier | | status | STRING | Status of the operation | Status of the operation | | operationDetailsList | | Details of the the submitted operation | Details of the the submitted operation | **Additional Information:** - On error, the system logs the failure, updates the job status to FAILED. User can retry the operation using retry API by providing job id - Admin user can retry the FAILED jobs using retry API or can cancel the job using cancel API ## Get Update Metadata Definition Jobs API This endpoint is to retrieve a paginated list of retrofit jobs for a specific tenant. It supports optional filtering by job status. If no status is provided, all jobs are returned. **Endpoint:** `GET (/api/datalake/v4/retrofit-jobs)` **Request:** `No request body` **Query Parameters:** The overview of the parameters, datatypes and detailing paths are displayed in the table: | Query Parameters | Datatype | Description | | ---------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | page | INTEGER | Page number for pagination. - Default value is 0. | | size | INTEGER | Specifies the number of elements in a page. **Example:** The default value of page and size is 0 and 10 respectively. The max value of size can be 200. To access the more jobs one can call the API with incremented page number. **/metadataDefinitionUpdateJobs?page=1&size=200** The above URI will display the next 200 jobs. For more jobs, page number has to be incremented. Default value : 10 | | status | STRING | Filter jobs with status (IN_PROGRESS, COMPLETED, FAILED, CANCELLED) | **Response:** ```json [ { "metadataDefinitionUpdateJobs": [ { "jobId": "2aede1a7-9378-4245-9b89-921888a8953e", "status": "FAILED", "created": "2025-07-28T14:53:26Z", "updated": "2025-07-30T11:42:13Z", "path": "data/ten=qvstnt3/", "failureReason": "This request is not authorized to perform this operation.", "retryCount": 0, "operationDetailsList": [ { "key": "country_of_origin", "metadataCollectionId": "document_review", "operationType": "SEARCHABLE_TO_NONSEARCHABLE", "usedInSDSPolicy": false, "additionalDetails": [ { "fieldName": "StringType", "fieldValues": [ "ALPHABETIC" ] } ] } ] } ], "page": { "size": 0, "totalElements": 0, "totalPages": 0, "number": 0 } } ] ``` ## Get Update Metadata Definition Job by ID API This endpoint is to retrieve the detail of update metadata definition job for the provided job ID. **Endpoint:** `GET /metadataDefinitionUpdateJobs/{id}` **Request:** `No request body` **Response:** ```json { "jobId": "2aede1a7-9378-4245-9b89-921888a8953e", "status": "FAILED", "created": "2025-07-28T14:53:26Z", "updated": "2025-07-30T11:42:13Z", "path": "data/ten=qvstnt3/", "failureReason": "This request is not authorized to perform this operation.", "retryCount": 0, "operationDetailsList": [ { "key": "country_of_origin", "metadataCollectionId": "document_review", "operationType": "SEARCHABLE_TO_NONSEARCHABLE", "usedInSDSPolicy": false, "additionalDetails": [ { "fieldName": "StringType", "fieldValues": [ "ALPHABETIC" ] } ] } ] } ``` ## Retry Failed Update Metadata Definition Job API This endpoint retry FAILED jobs by providing the job ID. If the specified job ID exists, the retry will resume the update metadata definition job from where it failed. Retry operation is only allowed for jobs with FAILED status. **Endpoint:** `POST /metadataDefinitionUpdateJobs/{id}/retry` **Request:** `No request body` **Response:** ```json { "jobId": "0b68f101-4016-42cc-b511-05774f2f47a1", "status": "IN_PROGRESS", "created": "2025-07-28T14:53:26Z", "updated": "2025-07-30T11:42:13Z", "path": "data/ten=qvstnt3/", "operationDetailsList": [ { "key": "country_of_origin", "metadataCollectionId": "document_review", "operationType": "SEARCHABLE_TO_NONSEARCHABLE", "usedInSDSPolicy": false, "additionalDetails": [ { "fieldName": "StringType", "fieldValues": [ "ALPHABETIC" ] } ] } ] } ``` ## Cancel Failed Update Metadata Definition Job API This endpoint cancel FAILED jobs by providing the job ID. Cancel operation is only allowed for jobs with FAILED status. Note If the job goes in FAILED state after multiple retry, cancel API can be used to move the job to CANCELLED status. It is important to move the FILED job to CANCELLED state as it will hold up the state of the job to retry **Endpoint:** `POST /metadataDefinitionUpdateJobs/{id}/cancel` **Request:** `No request body` **Response:** ```json { "jobId": "0b68f101-4016-42cc-b511-05774f2f47a1", "status": "CANCELLED", "created": "2025-07-28T14:53:26Z", "path": "data/ten=qvstnt3/", "operationDetailsList": [ { "key": "country_of_origin", "metadataCollectionId": "document_review", "operationType": "SEARCHABLE_TO_NONSEARCHABLE", "usedInSDSPolicy": false, "additionalDetails": [ { "fieldName": "StringType", "fieldValues": [ "ALPHABETIC" ] } ] } ] } ``` # Uploading data to Integrated Data Lake This section describes how to upload the data to Integrated Data Lake. ## Prerequisites The selection of methods solely depends on the requirement. You can upload the data to Integrated Data Lake using below defined methods: 1. Generate signed URL for AWS or Shared Access Signatures (SAS) for Azure 1. Cross account access for AWS or Service Principal for Azure 1. Simple Token Service for AWS ### Generate Signed URL or Shared Access Signatures To generate signed URL or Shared Access Signatures to upload an object, follow below steps: - **Endpoint**: ```http POST /api/datalake/v3/generateUploadObjectUrls ``` - **Content-Type**: application/json - **Request example**: ```json { "paths": [ { "path": "myfolder/mysubfolder/myobject.objext" } ] } ``` ```json { "paths": [ { "path": "myfolder/mysubfolder/myobject.objext" } ] } ``` ```json { "paths": [ { "path": "myfolder/mysubfolder/myobject.objext" } ] } ``` - **Response example:** ```json { "objectUrls":[ { "signedUrl":"https://datalake-integ-dide2-5234525690573.s3.eu-central-1.amazonaws.com/data/ten%3Ddide2/myfolder/mysubfolder/myobject.objext?X-Amz-Security-Token=Awervzdg23452xvbxd3434ddg&X-Amz-SignedHeaders=host&X-Amz-Expires=7200&X-Amz-Credentials=ASIATCES50453sdf&X-Amz-Signature=2e2342sfgsdfgsdgh", "path":"myfolder/mysubfolder/myobject.objext" } ] } ``` ```json { "objectUrls":[ { "Shared Access Signatures":"https://idltntprovisioningrc.blob.core.windows.net/datalake-rc-punrc118/data/ten=punrc118/folder1/mysensordata.log?sv=2018-11-09&spr=https&se=2020-03- 06T12%3A57%3A24Z&sr=b&sp=aw&sig=Hz154Gtw6xkZzD5MB7BaiFNx9mcU0ZqswEVdJfcWTUg%3D&2018-01-01T00%3A00%3A00.0000000Z", "path":"folder1/mysensordata.log" } ] } ``` ```json { "objectUrls": [ { "signedUrl": "https://datalake-integ-cdiot0-1627437476734.oss-cn-shanghai.aliyuncs.com/data/ten%3Dcdiot0/myfolder/mysubfolder/myobject.objext?Expires=1661144053&OSSAccessKeyId=LTAI5t9E24F6VDb6Q4hsyFNk&Signature=sz8e4JG42tQIX3Jg%2BNU1ZPRtlUQ%3D", "path": "myfolder/mysubfolder/myobject.objext" } ] } ``` You can use this signed URL to download one or multiple objects from the target folder. This URL is valid for 120 mins. Once the time limit is expired, you need to generate a new signed URL. **Request Sample**: ```http PUT https://datalake-integ-dide2-5234525690573.s3.eu-central-1.amazonaws.com/data/ten%3Ddide2/myfolder/mysubfolder/myobject.objext?X-Amz-Security-Token=Awervzdg23452xvbxd3434ddg&X-Amz-SignedHeaders=host&X-Amz-Expires=7200&X-Amz-Credentials=ASIATCES50453sdf&X-Amz-Signature=2e2342sfgsdfgsdgh ``` ```http PUT https://idltntprovisioningrc.blob.core.windows.net/datalake-rc-punrc118/data/ten=punrc118/folder1/mysensordata.log?sv=2018-11-09&spr=https&se=2020-03-06T12%3A57%3A24Z&sr=b&sp=aw&sig=Hz154Gtw6xkZzD5MB7BaiFNx9mcU0ZqswEVdJfcWTUg%3D&2018-01-01T00%3A00%3A00.0000000Z ``` ### Cross account access for AWS only This method can be used if you need a continuous access to the desired folder for upload. Consider an example where you have an AWS account in which any application resides. This application needs to continuously access IDL folder. In such scenarios, Cross Account Access is suitable. To use this method, you can follow below steps: 1. To create cross account on which access needs to be provided, ```http POST /crossAccounts Content-Type: application/json ``` **Request example:** ```json { "name": "testCrossAccount", "accessorAccountId": "960568630345", "description": "Cross Account Access for Testing", "subtenantId": "204a896c-a23a-11e9-a2a3-2a2ae2dbcce4" } ``` **Response example:** ```json { "id": "20234sd34a23a-11e9-a2a3-2a2sdfw34ce4", "name": "testCrossAccount", "accessorAccountId": "960768132345", "description": "Cross Account Access for Testing", "timestamp": "2019-09-06T21:23:32.000Z", "subtenantId": "204a896c-a23a-11e9-a2a3-2a2ae2dbcce4", "eTag": 1 } ``` 1. Once the cross account is created, perform cross account accesses to provide the desired access on desired prefix. ```text x-ms-blob-type: BlockBlob ``` ```http POST /crossAccounts/20234sd34a23a-11e9-a2a3-2a2sdfw34ce4/accesses Content-Type: application/json ``` **Request example:** ```json { "description": "Access to write to mysubfolder", "path": "myfolder/mysubfolder", "permission": "WRITE" } ``` **Response example:** ```json { "id": "781c8b90-c7b6-4b1c-993c-b51a00b35be2", "description": "Access to write to mysubfolder", "storageAccount": "dlbucketname", "storagePath": "data/ten=tenantname/myfolder/mysubfolder", "path": "myfolder/mysubfolder", "permission": "WRITE", "status": "ENABLED", "timestamp": "2019-11-04T19:19:25.866Z", "eTag": 1 } ``` 1. Once the access is provided, you can upload data through CLI or using AWS SDK to the desired prefix, with the relevant accesses. Follow the commands given below to upload the files to S3 bucket: `$ aws s3 cp myobject.objext s3://tgsbucket`\ `upload: ./myobject.objext to s3://tgsbucket/myobject.objext` # Working with Cross Account Accesses Note This section is applicable only for region Europe 1. You need this method for a continuous access to the desired folder for upload. Consider an example where you have an AWS account, where any application resides and this application needs to continuously access IDL folder. In such scenarios, Cross Account Access is useful. While using cross account access, adding one condition while uploading the file in the header gives full control to bucket owner. If this header is not included in the request, then the request fails with "Access Denied" error. **Header details:** `s3:x-amz-acl": "bucket-owner-full-control` **Command to add Header** From aws console, there is no option to provide this additional request parameter. However, it can be sent via cli or sdk mode. The below screenshot shows how to use it via cli. To use this method, you can follow below steps: 1. To create cross account on which access needs to be provided, use the following endpoint: ```http POST /crossAccounts ``` ```http Content-Type: application/json ``` **Request example:** ```json { "name": "testCrossAccount", "accessorAccountId": "960568630345", "description": "Cross Account Access for Testing", "subtenantId": "204a896c-a23a-11e9-a2a3-2a2ae2dbcce4" } ``` **Response example:** ```json { "id": "0234sd34a23a-11e9-a2a3-2a2sdfw34ce4", "name": "testCrossAccount", "accessorAccountId": "960768132345", "description": "Cross Account Access for Testing", "timestamp": "2019-09-06T21:23:32.000Z", "subtenantId": "204a896c-a23a-11e9-a2a3-2a2ae2dbcce4", "eTag": 1 } ``` 1. To get the list of Cross Accounts, use the following endpoint: ```http GET /crossAccounts ``` ```http Content-Type: application/json ``` **Response example:** ```json { "crossAccounts": [ { "id": "0234sd34a23a-11e9-a2a3-2a2sdfw34ce4", "name": "testCrossAccount", "accessorAccountId": "960768132345", "description": "Cross Account Access for Testing", "timestamp": "2019-09-06T21:23:32.000Z", "subtenantId": "204a896c-a23a-11e9-a2a3-2a2ae2dbcce4", "eTag": 1 } ], "page": { "size": 1, "totalElements": 1, "totalPages": 1, "number": 1 } } ``` 1. To get the details of the selected cross account, use the following endpoint: ```http GET /crossAccounts/0234sd34a23a-11e9-a2a3-2a2sdfw34ce4 ``` ```http Content-Type: application/json ``` **Response example:** ```json { "id": "0234sd34a23a-11e9-a2a3-2a2sdfw34ce4", "name": "testCrossAccount", "accessorAccountId": "960768132345", "description": "Cross Account Access for Testing", "timestamp": "2019-09-06T21:23:32.000Z", "subtenantId": "204a896c-a23a-11e9-a2a3-2a2ae2dbcce4", "eTag": 1 } ``` 1. To update the selected cross account, use the following endpoint: ```http PATCH /crossAccounts/0234sd34a23a-11e9-a2a3-2a2sdfw34ce4 ``` ```http Content-Type: application/json ``` **Request example:** ```json { "name": "updatedTestCrossAccount", "description": "Updated Cross Account Access for Testing", } ``` **Response example:** ```json { "id": "0234sd34a23a-11e9-a2a3-2a2sdfw34ce4", "name": "updatedTestCrossAccount", "accessorAccountId": "960768132345", "description": "Updated Cross Account Access for Testing", "timestamp": "2019-09-06T21:25:32.000Z", "subtenantId": "204a896c-a23a-11e9-a2a3-2a2ae2dbcce4", "eTag": 1 } ``` 1. To delete the selected cross account, use the following endpoint: ```http DELETE /crossAccounts/0234sd34a23a-11e9-a2a3-2a2sdfw34ce4 ``` **Response example:** ```http 204 Deleted ``` 1. Once the cross account is created, create cross account accesses to provide the desired access on desired prefix. This can be done by using teh following endpoint: ```http POST /crossAccounts/20234sd34a23a-11e9-a2a3-2a2sdfw34ce4/accesses ``` ```http Content-Type: application/json ``` **Request example:** ```json { "description": "Access to read to mysubfolder", "path": "myfolder/mysubfolder", "permission": "READ" } ``` **Response example:** ```json { "id": "781c8b90-c7b6-4b1c-993c-b51a00b35be2", "description": "Access to read to mysubfolder", "storageAccount": "dlbucketname", "storagePath": "data/ten=tenantname/myfolder/mysubfolder", "path": "myfolder/mysubfolder", "permission": "READ", "status": "ENABLED", "timestamp": "2019-11-04T19:19:25.866Z", "eTag": 1 } ``` 1. To get details of the cross account accesses, use the following endpoint: ```http GET /crossAccounts/20234sd34a23a-11e9-a2a3-2a2sdfw34ce4/accesses ``` ```http Content-Type: application/json ``` **Response example:** ```json { "crossAccountAccesses": [ { "id": "781c8b90-c7b6-4b1c-993c-b51a00b35be2", "description": "Access to read to mysubfolder", "storageAccount": "dlbucketname", "storagePath": "data/ten=tenantname/myfolder/mysubfolder", "path": "myfolder/mysubfolder", "permission": "READ", "status": "ENABLED", "timestamp": "2019-11-04T19:19:25.866Z", "eTag": 1 } ], "page": { "size": 1, "totalElements": 1, "totalPages": 1, "number": 1 } } ``` 1. To get details of the selected cross account accesses, use the following endpoint: ```http GET /crossAccounts/20234sd34a23a-11e9-a2a3-2a2sdfw34ce4/accesses/781c8b90-c7b6-4b1c-993c-b51a00b35be2 ``` ```http Content-Type: application/json ``` **Response example:** ```json { "id": "781c8b90-c7b6-4b1c-993c-b51a00b35be2", "description": "Access to read to mysubfolder", "storageAccount": "dlbucketname", "storagePath": "data/ten=tenantname/myfolder/mysubfolder", "path": "myfolder/mysubfolder", "permission": "READ", "status": "ENABLED", "timestamp": "2019-11-04T19:19:25.866Z", "eTag": 1 } ``` 1. To update the created cross account access, use the following endpoint: ```http PATCH /crossAccounts/20234sd34a23a-11e9-a2a3-2a2sdfw34ce4/accesses/781c8b90-c7b6-4b1c-993c-b51a00b35be2 ``` ```http Content-Type: application/json ``` **Request example:** ```json { "description": "Access to read to mysubfolder", "status": "ENABLED" } ``` **Response example:** ```json { "id": "781c8b90-c7b6-4b1c-993c-b51a00b35be2", "description": "Access to read to mysubfolder", "storageAccount": "dlbucketname", "storagePath": "data/ten=tenantname/myfolder/mysubfolder", "path": "myfolder/mysubfolder", "permission": "READ", "status": "ENABLED", "timestamp": "2019-11-04T19:19:25.866Z", "eTag": 1 } ``` 1. To delete the created cross account access, use the following endpoint: ```http DELETE /crossAccounts/20234sd34a23a-11e9-a2a3-2a2sdfw34ce4/accesses/781c8b90-c7b6-4b1c-993c-b51a00b35be2 ``` **Response example:** ```http 204 deleted ``` 1. Once the accesses is provided, you can upload data through CLI or using AWS SDK to the desired prefix with the relevant accesses. Follow these commands to upload the files to S3 bucket: `$ aws s3 cp myobject.objext s3://tgsbucket`\ `upload: ./myobject.objext to s3://tgsbucket/myobject.objext` Follow these commands to download the files from S3 bucket: `$ aws s3 cp s3://tgsbucket/myobject.objext .`\ `download: s3://tgsbucket/myobject.objext to ./myobject.objext` # Upload large files in Integrated Data Lake The Integrated Data Lake provides an API interface to upload files via Insights Hub Gateway. Currently, API restricts the user from uploading the files larger than 100 MB. This presents a challenge by hindering the seamless transfer and storage of large data files, by impacting user experience and restricting the system's capacity to address a variety of data types and user requirements. A multipart upload allows an application to upload a large file in smaller parts. After uploading, the application combines the smaller parts into the original larger file. The advantages of uploading a large file in smaller parts are: - Improves the performance by uploading many smaller parts in parallel. - Recovers from a network error more quickly by restarting the upload for the failed parts. Note Once multipart request is initiated for a file, - File with the same name cannot be uploaded (single file upload or another multipart request). - Metadata operations (add or update) on the file will be blocked till multipart request is completed. - File with the same name cannot be deleted from the storage. To upload a large file in Integrated Data Lake, follow these steps: 1. Initiate the multipart upload and obtain an `uploadId` via the POST API call. - The user initiates the multipart request, valid only for 24 hours. All the parts related to the file are uploaded in 24 hours. After 24 hours, multipart request gets expired and the uploading is performed again. - `uploadId` is provided to the user. All the parts are uploaded only under "uploadId", valid only for 24 hours. - Once the multipart is initiated, `timeout` is provided in the response which is configured only for 24 hours. **Endpoint:** ```http POST /multipart ``` **Request example:** ```json { "metadataKeyValues": [ { "key": "country_of_origin", "metadataCollectionId": "document_review", "values": [ "IN" ] } ] } ``` **Response example:** ```json { "path": "data/ten=punint01/sp165/bigfile.log", "uplaodId": "5e36d55270074d65acb1393463c30a86", "timeout": "2024-01-19 09:01:45", "status": "IN_PROGRESS" } ``` 1. Divide the large file into multiple parts and upload the parts of a large file in parallel via the PUT API call. - Once the multipart is initialized, user can upload the parts under the provided `uploadId` in the response of Initiate Upload Request. - The single part should be a minimum of 50 MB in size and a maximum up to 100MB. - A maximum of 10,000 parts can be uploaded in a single multipart request. - Every part when uploaded successfully, returns `partCommitId` in response. **Endpoint**: ```http PUT /multipart/{uploadId} ``` **Request example:** ```json { "metadata": { "metadataKeyValues": [ { "key": "country_of_origin", "metadataCollectionId": "document_review", "values": [ "IN" ] } ] }, "content": "string" } ``` **Response example:** ```json { "name": "test1.csv", "path": "/folder1/test1/test1.csv", "location": "ten=punvs1/folder1/test1/test1.csv", "lastModified": "2018-10-03T09:21:36.559Z", "size": 25, "metadataKeyValues": [ { "key": "country_of_origin", "metadataCollectionId": "document_review", "values": [ "IN" ], "isPropagated": true } ] } ``` - **List multipart file upload requests**: This GET method is used to list all the multipart requests that are initiated along with their status. **Endpoint:** ```http GET /multipart ``` **Sample Response:** ```json { "multipartRequests": [ { "path": "/floor1/machine/logs.txt", "uploadId": "string", "status": "COMPLETED" } ] } ``` - **List uploaded parts within a multipart request**: This GET method is used to list all the parts upload status for the given "uploadId" along with the status of the uploadRequest, parts for which upload is completed. **Endpoint:** ```http GET /multipart/{uploadId} ``` **Sample Response:** ```json { "path": "/floor1/machine/logs.txt", "status": "COMPLETED", "parts": [ { "partNumber": 1, "partCommitId": "string" } ] } ``` 1. Complete the upload by calling the POST API call to mark the completion of the upload. - Once all the parts are uploaded, user can call complete multipart API to close the multipart request. - User can provide the list of parts that combines to a single file along with `partCommitId`. - Only complete request parts are considered for combining the parts. Note Since only 10,000 parts are allowed, the maximum file size that can be uploaded could be restricted to 800 to 900 GB. **Endpoint:** ```http POST /multipart/{uploadId}/complete ``` **Sample Request:** ```json { "parts": [ { "partNumber": 1, "partCommitId": "string" } ] } ``` **Sample Response:** ```json { "name": "test1.csv", "path": "/folder1/test1/test1.csv", "location": "ten=punvs1/folder1/test1/test1.csv", "lastModified": "2018-10-03T09:21:36.559Z", "size": 25, "metadataKeyValues": [ { "key": "country_of_origin", "metadataCollectionId": "document_review", "values": [ "IN" ], "isPropagated": true } ] } ``` 1. Abort the upload by terminating the multipart file upload process. **Endpoint:** ```http DELETE /multipart/{uploadId} ``` Note Multipart request cannot be terminated if it is expired. Metadata management is a crucial aspect of data management within any Data Lake application. It involves the systematic organization, storage, retrieval, and maintenance of metadata, which is essentially data about data. This includes information about the characteristics, origin, usage, and relationships of the actual data stored in the Data Lake. Metadata within Integrated Data Lake was available in the form of tags. User could provide this tag based metadata on folders or on objects. Tags were single-word that provide a concise way to categorize content, focusing on key concepts or themes. It was a flexible way to organize the resources but they lack a standardized structure, potentially leading to ambiguity. While tags facilitate general categorization, their loose structure might not be as efficient for targeted searches or retrieval of specific information. With Metadata Management V4 APIs, transitioning from metadata as tags to key-value pair metadata brings about a shift from a more generalized categorization approach to a structured, detailed, and context-rich representation of data attributes. This change enhances precision, searchability, and overall data management capabilities in diverse and complex information ecosystems. The Metadata Management V4 APIs bring forth a range of new features to enhance the functionality of the Data Lake application: - **Metadata as Key-Value Pairs**: Introduces a structured approach to metadata by utilizing key-value pairs. Users can configure metadata keys, metadata collections, and define rules that govern the behavior of these collections. - **Enhanced Security for Object Uploads and Folder Creation**: Offers a more secure method for uploading objects or creating folders in the storage through the API Gateway. This is a notable improvement from the initial Integrated Data Lake, which initially supported only native interfaces for uploading and downloading objects. - **Metadata Inclusion during Object Upload and Folder Creation**: Allows users to provide metadata information concurrently with the upload of an object or the creation of a folder. This ensures that metadata is associated with data at the time of its addition to the Data Lake, facilitating comprehensive and real-time information management. # IoT File Service – API Overview Region Europe 1 New [Download OpenAPI Specification](https://documentation.mindsphere.io/insights-hub/api_specs/iotfile-v4-1-0-xcelerator.yaml) Old [Download OpenAPI Specification](https://documentation.mindsphere.io/insights-hub/api_specs/iotfile-v3-7-0-eu1-xcelerator.yaml) # IoT File Service – Bulk Deletion Samples ## Create Time To Live Rule for File deletion This API can be used by the users to create Time To Live(TTL) rules, which initiates bulk deletion of the files. Create TTL rule using the following endpoint: ```http POST/fileDeleteRules ``` Sample Request: ```json [ { "assetId": "0fcd06e781dd4bb0b9bd271d648741aa", "path": "/folder1/folder2", "deleteAfterDays": 30 } ] ``` Parameter Details: - assetId: Unique identifier of the asset for which the files needs to be deleted - path: Path under which rule will be executed and files would be deleted - deleteAfterDays: Number of days after which files would be deleted (min 30 & max365) It is calculated from last update date of the file Sample Response: ```json [ { "id": "0fcd06e7-81dd-4bb0-b9bd-271d648741aa", "assetId": "089336872ce342aba1a847f53d515dc7", "path": "/folder1/folder2", "deleteAfterDays": 30, "created": "2023-03-15 12:29:31.0" } ] ``` Limitations on creation of TTL rules: - Maximum 50 TTL rules can be created per tenant. This limit is configurable and can be increased as per requirement - Only one rule per hierarchy of the folder can be created. It is recommended to apply rule on asset, so that all the underlying files are deleted with the rule - Rule will be executed on the next day of creation. Generally rule is executed at 12AM UTC time. - Number of days is restricted to minimum 30 and maximum 365 ## Retrieving TTL rules This API gives the overall TTL rules created for file deletion on all the assets of the tenant. Retrieve TTL rules using the below endpoint: ```http GET/fileDeleteRules/ ``` This API gives the overall state of the rules. Description of TTL rules states is as shown below: | State | Description | | ACTIVE | Rule is create and is in active state. This rule will be executed daily | | MARKED_FOR_DELETE | This Rule is deleted but cannot be removed due to some pending execution related activities. Once the activities are completed, Rule will be removed automatically. No further operations are possible on this Rule | Sample Response: ```json { "fileDeleteRules": { "id": "0fcd06e7-81dd-4bb0-b9bd-271d648741aa", "assetId": "089336872ce342aba1a847f53d515dc7", "path": "abc33/", "deleteAfterDays": 1, "state": "ACTIVE", "created": "2020-03-01 09:12:28.110" }, "page": { "size": 0, "totalElements": 0, "totalPages": 0, "number": 0 } } ``` ## Retrieving Rule Details This API gives the overall TTL rules created for file deletion of the given id. Retrieve TTL rules for the given id using the below endpoint: ```http GET/fileDeleteRules/{id} ``` Sample Response: ```json { "id": "0fcd06e7-81dd-4bb0-b9bd-271d648741aa", "assetId": "0fcd06e781dd4bb0b9bd271d648741aa", "path": "/folder1/folder2", "deleteAfterDays": 30, "state": "MARKED_FOR_DELETE", "created": "2020-03-01 09:12:28.110" } ``` ## Delete TTL rule This API deletes the already created file delete rule for a given id. If a TTL rule is deleted, its next scheduled execution will not be triggered. Delete TTL rule for the given id using the below endpoint: ```http DELETE/fileDeleteRules/{id} ``` # IoT File Service ## Idea The IoT File Service provides the user with file management for files related to assets. The service provides an interface to manage data, search files and download them to specified agents. Files can also be uploaded to a temporary storage and managed there. ## Access For accessing this service, you need to have the respective roles listed in [IoT File Service roles and scopes](https://documentation.mindsphere.io/MindSphere/concepts/concept-roles-scopes.html#iot-file-service). A user can only interact with files within their environment and subtenants. For accessing the Secure Data Sharing (SDS) protected APIs, you need to have appropriate Policy Definitions in place. For the list of supported APIs and Required Actions, refer to the [Required Policy actions by Industrial IoT services](../core-resourceaccessmanagement/api-resourceaccessmanagement-services-actions-map.html#iot-file-service) section. ## Basics The service implements CRUD (create, read, update and delete) functionality and searching for files. Each file is linked to an asset. An asset can have multiple files attached. The asset ID is required for each request. Each file has a set of standard properties as metadata: - File ID - File name - File path - File type (e.g., image) - File size - Description - Timestamps (created, updated) The content of these files is not parsed by Insights Hub and it requires custom applications or analytical tools to interpret and visualize the data. The maximum request payload that can be stored or retrieved depends on the client's network speed. The current timeout is one minute, and any file that takes longer than one minute to read or write results in an error 502 "Bad Gateway". If a file is too large to upload in one minute, use the multi part upload feature. ### Supported Use Cases Typical use cases that are supported within File Service are as follows: - Storing static data related to Assets (configuration files, images, specifications, etc.) - Supports low frequency operations on the data (once in a day, weekly or even monthly) - Temporary storage of data to explore real time data storage in Time Series Note Avoid the use of File Service for high frequency and continuous read or write, analytical workflows and storing high volume real-time data (Time Series data). ### Versioning The service does not provide explicit versioning. If different versions of a file need to be stored, put a version identifier in the file name to make it unique. ### File Deletion Files are always deleted physically. There is no logical delete. Files of an asset are automatically deleted if the asset is deleted physically. Files can be deleted from the storage in the following ways: | **Delete Mechanism** | **Description** | | -------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Single File Delete | Delete a file for the specified asset (entity) and path (actual file + metadata). This is a synchronous call. | | Bulk Delete | Delete files in bulk (actual files + metadata). This is an asynchronous activity in which files are deleted from storage within a day. There is an API available for bulk deletion using Time To (Live) rule. Users are required to create a Time To Live (TTL) rule to initiate bulk file deletion. The TTL rule specifies the duration after which files are automatically deleted by the system. This rule operates on a daily basis, identifying and removing files that meet the criteria outlined in the rule. | | Asset Delete | If the underlying asset is deleted, the files related to that asset are also deleted from the storage. File deletion is triggered on asset delete notification. This is also an asynchronous activity. | ### File Search Files can be queried using filtering and sorting operations. For more information, refer to the [Filtering documentation](api-iotfile-references-filtering.html). ## Features The IoT File Service exposes its API for realizing the following tasks: - Create a new file - Update (overwrite) an existing file - Read a file - Delete a file - Search for files by metadata - Count and order files by metadata - Multi part upload and download - Bulk delete of files for an asset ## Limitations - For security reasons, a strict timeout is enforced. Currently, this leads to the situation that the size of a file which can be uploaded/ downloaded depends on the network bandwidth at client side. - When uploading files to the File Service, a file name cannot contain blank spaces. Blank spaces in the file name will prevent the file form being stored. - After deleting an asset, associated files might still be accessible for up to 30 minutes as this data is deleted asynchronously. - After deleting a file, it will take some time (up to 60 - 120 seconds) to create the file with the same name again depending on the data stored in storage. - After creating or deleting the file, it will take some time (up to 3 seconds) to get the file in "search file" API. - In File Service, a maximum of 100 characters are allowed for a file name. - Using special characters like '\*' or '?' can delay the search and might result in slow search performance. All requests pass through [Industrial IoT Gateway](https://documentation.mindsphere.io/MindSphere/concepts/concept-gateway-url-schemas.html) and must adhere to the [Industrial IoT Gateway Restrictions](https://documentation.mindsphere.io/MindSphere/concepts/concept-gateway-url-schemas.html#restrictions). ## Usage Quota and Limits IoT File Service APIs are restricted with following limitations: | **API Endpoint** | **Quota Parameter** | **XS (Offering ID5302)** | **S (Offering ID5303)** | **M (Offering ID5304)** | **L (Offering ID5305)** | **XL (Offering ID5306)** | | ----------------------------------------- | -------------------------------------------- | ------------------------ | ----------------------- | ----------------------- | ----------------------- | ------------------------ | | PUT (Short Window) | core_iotfile_v3_files--\_put_30s | 8 | 10 | 12 | 14 | 16 | | PUT (Long Window) | core_iotfile_v3_files--\_put_600s | 100 | 140 | 180 | 220 | 280 | | LIST (Short Window) | core_iotfile_v3_files--\_get_30s | 10 | 12 | 15 | 18 | 20 | | LIST (Long Window) | core_iotfile_v3_files--\_get_600s | 210 | 240 | 270 | 300 | 350 | | GET (Short Window) | core_iotfile_v3_files-~\_get_30s | 12 | 14 | 18 | 22 | 25 | | GET (Long Window) | core_iotfile_v3_files-~\_get_600s | 200 | 250 | 300 | 400 | 500 | | DELETE (Short Window) | core_iotfile_v3_files--\_delete_30s | 1 | 2 | 3 | 4 | 5 | | DELETE (Long Window) | core_iotfile_v3_files--\_delete_600s | 10 | 12 | 15 | 20 | 25 | | POST Bulk Delete (Short Window) | core_iotfile_v3_filesdeletejobs_post_6h | 1 | 2 | 3 | 4 | 5 | | POST Bulk Delete (Long Window) | core_iotfile_v3_filesdeletejobs_post_24h | 2 | 3 | 4 | 6 | 8 | | Get Bulk Delete Jobs (Short Window) | core_iotfile_v3_files-deletejobs-~\_get_30s | 1 | 2 | 3 | 4 | 5 | | Get Bulk Delete Jobs (Long Window) | core_iotfile_v3_files-deletejobs-~\_get_600s | 8 | 9 | 10 | 12 | 15 | | Get Bulk Delete Job Detail (Short Window) | core_iotfile_v3_filesdeletejobs_get_30s | 1 | 2 | 3 | 4 | 5 | | Get Bulk Delete Job Detail (Long Window) | core_iotfile_v3_filesdeletejobs_get_600s | 8 | 9 | 10 | 12 | 15 | Note Rate Limits can cause discontinuity in the use case flow. Ensure to check the rate limits before implementing File Service APIs. Also, check the alternate storage mechanisms within Insights Hub for advanced use cases. ## Example Scenario The application developer of a brewery wants to store files associated to assets in order to provide additional meta information (e.g., images, technical descriptions, manuals, etc.). The developer uses the IoT File Service to upload files to an asset. ## Related Links - [Asset Management Service](../advanced-assetmanagement/api-assetmanagement-overview.html) # IoT File Service – Filtering ## Search Filter Operators The allowed operators in a filter clause are: - `eq` or `=` - `gte` or `>=` - `gt` or `>` - `lte` or `<=` - `lt` or `<` Filters can be combined using the and operator. Blanks in the wrong place and invalid parameters will lead to the filter being ignored. Filtering is possible on the following fields: - File update timestamp - File path - File name Note Avoid using wild card character (\*) in the search filter. This will have performance impact on the search queries. This will be restricted in the future. By default, GET requests for a list of files return only the first up to 500 files. Use the `offset` parameter in the query string of the GET request to adjust at which file to start the list. Use the `limit` parameter to adjust how many files to return. Be aware that the `offset` parameter is limited to 10,000. It is recommended to refine the search to limit the response to small result sets. ## Search Filter Syntax The following table lists examples for the search filter syntax for each operator: | Operator | Description | Operation | Example | | -------- | ---------------------------------------- | -------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------- | | eq | equal to specified value | | path eq 'my/path' | | lt | less than specified value | Restrict usage to comparing Date only. Using it for path or name is not preferable and will be blocked in the future | updated lt '2018-07-27T17:05:31.208Z' | | gt | greater than specified value | Restrict usage to comparing Date only. Using it for path or name is not preferable and will be blocked in the future | updated gt '2018-07-27T17:05:31.208Z' | | lte | less than or equal to specified value | Restrict usage to comparing Date only. Using it for path or name is not preferable and will be blocked in the future | updated lte '2018-07-27T17:05:31.208Z' | | gte | greater than or equal to specified value | Restrict usage to comparing Date only. Using it for path or name is not preferable and will be blocked in the future | updated gte '2018-07-27T17:05:31.208Z' | | and | combine two conditions | | name eq 'result\*.txt' and updated lte '2018-07-23T13:16:07.078Z' | ## Sorting Files Default sorting in File Service would be applicable on 'updated date'. No further sorting will be applicable for the APIs. Note Current support for sort by name (order=name asc/desc) and by path (order=path) will be blocked in the future in File Service APIs. # IoT File Service – Multi Part Operations ## Multi Part Upload Multi part upload is performed by the following steps: 1. [Initiate multi part upload](#initiate-the-file-upload) 1. [Upload file parts](#upload-file-parts) 1. [Complete multi part upload](#complete-the-upload) ### Initiate the File Upload Initiate a multi part upload by providing the `upload=start` query parameter to the upload request. **This initiation request and does not accept any file part**. The example request below initiates a multi part upload for a file `exampleFile` in the sub directory `optionalDirectory`: ```http {{gatewayUrl}}/api/iotfile/v3/files/{{entityId}}/optionalDirectory/exampleFile?upload=start ``` ### Upload File Parts Specify the part number as a query parameter when uploading file parts. The following rules apply for file parts: - Part numbers from 1 to 1,000 are accepted and can be uploaded asynchronously. - When all parts are uploaded, their part numbers must be sequential. - Each part must have a minimum size of 5 MB. - Parts belonging to incomplete file uploads are automatically deleted after 2 days. - If a part number is used twice, the existing part is overwritten. ```http {{gatewayUrl}}/api/iotfile/v3/files/{{entityId}}/optionalDirectory/exampleFile?part=1 ``` ### Complete the Upload Provide the request parameter `upload=complete` to complete a multi part upload. The example below completes a multi part upload for a file `exampleFile` in the sub directory `optionalDirectory`. The complete request accepts a file part that is smaller than the minimum allowed part size and treats it as the final part. ```http {{gatewayUrl}}/api/iotfile/v3/files/{{entityId}}/optionalDirectory/exampleFile?upload=complete ``` ## Abort a Multi Part Upload Abort a multi part upload using the `upload=abort` parameter. All information for the selected multi part upload is physically deleted. ```http {{gatewayUrl}}/api/iotfile/v3/files/{{entityId}}/optionalDirectory/exampleFile?upload=abort ``` ## Multi Part Update Specify the `If-Match` header parameter in each request of the [Multi Part Upload](#multi-part-upload) to update an existing file using multi part upload. ## List Current Multi Part Uploads List the current file parts for a specified file using a GET request to the `filelist` endpoint. Each item in the file part list contains: - `objectKey`: The file key consisting of {tenant storage prefix}/{entityId}/{filepath} - `partNo`: The specified multi part number - `created`: UTC time the part was uploaded displayed in milliseconds Sample request: ```http {{gatewayUrl}}/api/iotfile/v3/fileslist/{{entityId}}/optionalDirectory/exampleFile ``` Sample response: ```json [ { "objectKey": "iotteni/00189ab4c65243bb9d77f17332dd570a/optionalDirectory/exampleFile", "partNo": 1, "created": 1540509915000 }, { "objectKey": "iotteni/00189ab4c65243bb9d77f17332dd570a/optionalDirectory/exampleFile", "partNo": 2, "created": 1540509931000 } ] ``` ## Multi Part Download Sample Request: ```json gatewayUrl/api/iotfile/v3/files/entityId/optionalDirectory/exampleFile ``` You need to specify 'range' in the header with value as 'bytes=200-400'. Sample Response: ```json 206 Partial content of the file ``` Download a portion of an existing file by using the `Range` parameter and specifying the byte range to download, e.g.: `bytes=1-100` ## Bulk Delete Delete multiple files within the file storage for any specified asset. In one request, a maximum of 3 months of file data can be deleted. ### Request to delete file Sample Request: ```json { "assets": [ { "id": "089336872ce342aba1a847f53d515dc7", "path": "/logs", "timeFrom": "2020-03-01T09:12:28.110Z", "timeTo": "2020-06-01T09:12:28.110Z" } ] } ``` The `id` is a mandatory field and cannot be null or empty. The `timeFrom` and `timeTo` parameters are optional. The range from `timeFrom` and `timeTo` should not exceed 3 months. If these parameters are not provided then, the files created before 90 days of the submission of this request, will be deleted. Sample Response: ```json { "id": "a87d8d68-9d6d-42f8-9eb9-8899fa90bb3f", "timestamp": "2020-06-08T09:12:28Z", "status": "IN_PROGRESS" } ``` In addition, it is possible to view the status of the bulk delete operation. Sample Response: ```json { "id": "a87d8d68-9d6d-42f8-9eb9-8899fa90bb3f", "timestamp": "2020-06-08T09:12:28Z", "status": "COMPLETED_WITH_ERRORS", "assets": [ { "id": "089336872ce342aba1a847f53d515dc7", "path": "/", "timeFrom": "2020-03-01T09:12:28.110Z", "timeTo": "2020-06-01T09:12:28.110Z", "status": "FAILED", "filesToBeDeleted": 500, "filesDeleted": 400, "filesRemaining": 100, "error": "Something went wrong while deleting some files" } ] } ``` It is also possible to get all the bulk delete operations performed by an environment. Sample response: ```json { "deleteJobs": [ { "id": "a87d8d68-9d6d-42f8-9eb9-8899fa90bb3f", "timestamp": "2020-06-08T09:12:28Z", "status": "IN_PROGRESS" } ] } ``` The IoT File Service empowers users with efficient file management capabilities tailored for assets. This service offers a user-friendly interface for data management, file searching, and downloading files associated with specific assets or agents. As the usage of the File Service grows over time, it is essential to address storage limitations to ensure optimal performance. With the introduction of version 4.x, users gain access to the IoT File Service Bulk Delete API. This API allows users to systematically remove files from storage at regular intervals, aiding in the maintenance of service performance. To initiate bulk file deletion, users are required to create a Time To Live (TTL) rule. The TTL rule specifies the duration after which files are automatically deleted by the system. This rule operates on a daily basis, identifying and removing files that meet the criteria outlined in the rule The V4.x API is designed with a primary focus on delivering intelligent and enhanced performance compared to the current Bulk Delete APIs. Key features include: - The V4.x API streamlines the process of cleaning up file storage at regular intervals - Users benefit from an automated provision to delete files from the storage, simplifying the maintenance process, free up space and ensure that you have enough room for new files. - Users can easily configure criteria for file deletion, specifying the number of days as the chosen parameter - The API allows users to set up rules specifically for selected assets, providing a more tailored approach to file management - Users enjoy the flexibility of deleting files without any imposed restrictions on the number or size of files, offering a seamless and unrestricted experience - By regularly deleting files, you can improve the speed and performance of your device. Concurrently, given that this involves the deletion of files from storage, it is crucial to take certain considerations into account to prevent any unforeseen behavior. - Deletion operations are executed based on the last updated date of files in storage. Performing operations on older files that meet the deletion criteria may result in unpredictable outcomes, as the last updated date of the file would be altered - The file storage system operates on an eventual consistency model. Although there may be temporary inconsistencies, both files and metadata will eventually attain a consistent state - Any modifications made to the Time To Live (TTL) rule will only take effect during the subsequent rule execution cycle. Users should be aware that changes to the rule may not be immediately reflected We recommend users to make use of File Service Bulk Delete APIs to delete the files from the storage for improved performance. For information on how to use the 4.x API, refer to Bulk Delete Operation. ## Bulk Delete Operation ### Process flow #### Create TTL Rule The user can create TTL rule by providing the following details: - Specific path (asset) - Number of days after which data should be deleted # IoT Time Series Ingest Rate Estimator ## Idea The IoT Time Series Ingest Rate Estimator helps to interactively estimate the rate at which the data is received at IoT Time Series services. Time Series Inbound Bandwidth (KB/s) in Usage Transparency reflects the same Ingest rate consumed by the tenant. ## Ingest Rate Ingest Rate is the timeseries data sent to IoT Time Series service from assets (devices) or applications via MindConnect elements like IoT 2040, MindConnect Nano etc or via direct API calls of IoT PUT APIs over a period of time. Ingest Rate is calculated in KB/s. ## Basics - Assets: A digital representation of a machine or an automation system with one or multiple automation units (e.g. PLC) connected to Insights Hub. For example, a pump, motor, PLC, an entire tool machine, a production line, a robot, a crane, a car, a windmill and so on. - Aspect: Aspects are a data modeling mechanism for assets. Aspects groups the related data points based on their logical association. For example: The pump skid has an aspect "Energy_consumption" that contains the datapoints "power", "current", "voltage" etc. An aspect is specified in "Asset Manager" and its name should have conjunction to datapoints and a physical asset. An aspect can consist of several variables. - Variables: Variables are actual data properties of the Assets under aspect. This can be machine values like pressure, temperature, current, voltage etc. - Record: Collection of values of an aspect variable collected from a sensor at a given timestamp. - Payload: Collection of records sent in one IoT TimeSeries API call which may contain records for one asset or multiple assets. ## Estimation of Ingest Rate Time Series Ingest rate is derived from the data ingestion in Time Series PUT API's. It is calculated by identifying the number of variables within an Aspect and their type. Each aspect depending on the number of variables along with sending frequency and sampling frequency will determine the size of an individual record for a specific aspect. This size then determines the overall size and the amount of data expected over multiple aspects and asset combination to derive Ingest Rate. For more precise calculation, refer the [Ingest Rate Estimator](#ingest-rate-estimator-new) section. ## Example Scenario The ingest rate is the size of payload of data received by Time Series at a given second. If, for example, 100 KB payload is received by IoT Time Series API at one second and next for the next 9 seconds there is not data received and then again 100 KB is received and if the same cycle repeats for 1 minute, then average for that minute is 600 KB/ 60 sec which is 10 kbps. ## Ingest Rate Estimator (New) The new estimator (Beta Version) can estimate more accurate ingest rate due to granularity and inputs of variable name length, length of variable values for customer scenario. It is available as microsoft excel sheet. For information, refer to [Ingest rate estimator](Files/Ingest_Rate_Calculator_Beta.xlsx). Info Based on customer use-case, only cells marked in Green needs to be updated for estimation of ingest rate. ## Ingest Rate Estimator (Old) | Aspect Type and Rate | | | ------------------------------------------------------------------------- | --- | | Sampling Rate How often do you collect data from sensor into your nanobox | | | Transfer Rate How often you send data from nanobox to Insights Hub | | | Variables | | | --------------------- | --- | | Number of Doubles | | | Number of Integers | | | Number of Booleans | | | Number of Strings | | | Average String Length | | | Asset and Aspect | | | ---------------- | --- | | Number of Aspect | | | Number of Asset | | \[Calculate Ingest Rate\](javascript:avgRecordCal() ) | | | | ------------------------ | --- | | Ingest Rate per Aspect | | | Tenant level Ingest Rate | | Note - Data from files and bulk import is not considered for ingest rate estimation. - The above calculation is for estimation purposes only. It is expected to have variation in actual scenario due to various reasons. - If Aspect and Variables are enabled for qualitycode, additional variables extended by \_qc also gets ingested. You can consider qualitycode variable also in calculator for better accuracy. # IoT Time Series Service – API Overview ## Region Europe 1 [Download OpenAPI Specification](/insights-hub/api_specs/iottimeseries-v3-8-0-xcelerator.yaml) IoT Timeseries are restricted with following limits. | **API Endpoint** | **Unit** | **Timeframe** | **XS** | **S** | **M** | **L** | **XL** | | ---------------------------------------- | -------- | ------------- | ------ | ----- | ----- | ----- | ------ | | IoT TS Aggregate V4 - Get | Nos | 1 Minute | 700 | 1400 | 2800 | 2800 | 5600 | | IoT Timeseries - Patch | Nos | 1 Minute | 10 | 20 | 30 | 30 | 30 | | IoT Timeseries - Get | Nos | 1 Minute | 1500 | 1500 | 2100 | 3600 | 4800 | | IoT Timeseries Bulk (Bulk Import) - Post | Nos | 1 Minute | 50 | 100 | 150 | 200 | 400 | | IoT Timeseries Bulk (Bulk Import) - Get | Nos | 1 Minute | 250 | 500 | 1000 | 2000 | 4000 | | IoT Timeseries Bulk (Read) - Get | Nos | 1 Minute | 100 | 200 | 300 | 500 | 800 | | Timeseries Inbound Bandwidth (KB) | KB | 1 Second | 1000 | 1000 | 2000 | 3000 | 5000 | | Timeseries Read Bandwidth (KB) | KB | 1 Second | 3000 | 4000 | 5000 | 8000 | 10000 | Note Rate Limits can cause discontinuity in the use case flow. Ensure to check the rate limits before implementing APIs. # IoT Time Series Service – Async API Specification [Download Async API Specification](/insights-hub/api_specs/iottimeseries-async-v3-0-1.yaml) # IoT Time Series Service – API Overview ## Available API versions | Region | API version | | -------- | ------------------------------------------------------- | | Europe 1 | [3.1.0](api-iottimeseries-async-api-swagger-3-0-1.html) | # IoT Time Series Service ## Idea The IoT Time Series Service is used to create, read, update, and delete time series data. Time series data is stored against an asset and an aspect. A time series record consists of a timestamp, one or more values, and an optional quality indicator for each variable, which is defined by the aspect type. Within the IoT Time Series Service, you can store and query time series data with a precision of 1 millisecond. ## Access For accessing this service, you need to have the respective roles listed in [IoT Time Series Service roles and scopes](https://documentation.mindsphere.io/MindSphere/concepts/concept-roles-scopes.html#iot-time-series). A user can only read data within their environment, subtenants and shared assets. For accessing the Secure Data Sharing (SDS) protected APIs, you need to have appropriate Policy Definitions in place. For the list of supported APIs and Required Actions, refer to the [Required Policy actions by Industrial IoT services](../core-resourceaccessmanagement/api-resourceaccessmanagement-services-actions-map.html#iot-time-series-service) chapter. ### Prerequisites #### Ingesting Time Series Data from a Field Device An onboarded asset must be connected to Insights Hub and produce time series data. A valid mapping must exist from device data points to asset variables. For instructions, refer to [Uploading agent data](https://documentation.mindsphere.io/MindSphere/howto/howto-agent-upload-data.html#creating-a-data-point-mapping). #### Ingesting Time Series Data from an Application An asset with respective variables must exist. ## Basics A time series record consists of a timestamp, one or more values, and an optional quality indicator for each value, as defined in the aspect type definition. If there are multiple variables in the aspect, they are expected to arrive in the same payload. Note Writing a time series record with the same timestamp, asset and aspect as an existing one completely overwrites the old record. There is no versioning of data. The timestamp may be specified with millisecond precision. Measurement values must match the type that is defined in the aspect type. Time series data can be of type `int`, `long`, `double`, `boolean`, `string`, `big_string` (blobs), or `timestamp`. The maximum sizes of strings and big strings are defined in the aspect type, and can be up to 255 and 100,000 respectively. The maximum size of a time series record is 200,000 byte. Each data type is validated: longs, doubles, and timestamps using eight byte, integers using four byte, strings and big strings using two byte per character, and Booleans using one byte. In the aspect type definition, there is a `qualitycode` field, which indicates whether quality code data will accompany the time series data. If the `qualitycode` value is true for a variable (property), the name of the quality code property is the same as the variable (property) name, extended by a `_qc` suffix. For example, the quality code property for the variable `temperature` would be `temperature_qc`. Multiple records for an asset and aspect can be written in one call to the service. Records can be read and deleted by identifying the asset, aspect, and time range. When writing or deleting data, the default behavior of the service is to queue the incoming request and perform the physical writes to the underlying data store asynchronously. If a user writes time series data and immediately tries to read it back, it may not yet be present in the data store and it will not be returned. If a user deletes data and then reads that time range, the data may not have been physically deleted yet, and could be returned in the response. Requests for an asset are processed in the order they are received. ## Features The IoT Time Series Service exposes its API for realizing the following tasks: - Read time series data for an asset or aspect - Read time series data for a specified time range - Read the latest uploaded data - Create time series data for a single or multiple aspects - Overwrite existing time series data - Delete time series data for a single aspect of an asset within a given time range Important It is recommended to use GET Queries with Time range for better throughput and performance. ## Usage Quota and Limits The general limits for IoT timeseries is mentioned in [API rate limits](api-iottimeseries-apiratelimits.html) ### Delete Limitations The Iot Time Series DELETE endpoint has the following restrictions: - Time Series data can only be deleted 7 days after it was submitted. - The range of a delete request must be UTC hourly aligned. - The range of a delete request cannot be greater than 366 days. - The maximum number of delete requests per environment is 1 per hour. - For the specific time frames where delete operation is undergoing and data ingestion happens, there might be data inconsistencies, and vice versa. - For Private Cloud, Delete is an asynchronous process and it will take upto 1 day for deleting timeseries data. ## Best Practices & Recommendations ### Data Ingest Recommendations - Use normal [PUT endpoint](https://documentation.mindsphere.io/MindSphere/apis/iot-iottimeseries/api-iottimeseries-api.html) for ingesting near real-time timeseries data. This normal PUT endpoint works for timeseries data corresponding to single asset-aspect combination. Once you ingest timeseries data using this endpoint, data gets persisted within single digit seconds into the persistent layer so that, data can be queried for reporting/dashboarding/analytical purpose. - Use [Multi asset multi aspect PUT endpoint](https://documentation.mindsphere.io/MindSphere/apis/iot-iottimeseries/api-iottimeseries-api.html) for ingesting near real-time timeseries data when you club timeseries data for multiple asset-aspect combinations. Once you ingest timeseries data using this endpoint, data gets persisted within single digit seconds into the persistent layer so that, data can be queried for reporting/dashboarding/analytical purpose. - Use [bulk ingest endpoint](https://documentation.mindsphere.io/MindSphere/apis/iot-iottsbulk/api-iottsbulk-api.html) for ingesting historian data or data in bulk which do not have near real-time expectations. Once you ingest timeseries data using this endpoint, data gets persisted within 4 hours into the persistent layer so that, data can be queried for reporting/dashboarding/analytical purpose. - It's best practice to include error handling for 429 responses in your code. The 429 status code means too many requests. The Retry-After header specifies no. of seconds after which API call can be retried. Your code should stop making additional API requests until enough time has passed to retry. ### Data Query Recommendations - Requirement for fetching near real time data for specific time range up to 90 days of data from now. we have 2 API endpoints for it: - [Normal raw data GET endpoint](https://documentation.mindsphere.io/MindSphere/apis/iot-iottimeseries/api-iottimeseries-api.html) fetches maximum of 2000 records in one call with latency expectation of ~1 second. - [Bulk stream GET endpoint](https://documentation.mindsphere.io/MindSphere/apis/iot-iottsbulk/api-iottsbulk-api.html) fetches as many records as it can pull within 60 seconds (gateway timeout duration), hence no latency expectation. - Requirements for fetching data older than 90 days to maximum of 1 year old data, we can use [aggregate API endpoint](https://documentation.mindsphere.io/MindSphere/apis/iot-iottsaggregates/api-iottsaggregates-api.html) with right input parameter for interval value & interval unit. ### Generic Recommendations - For live data ingestion use cases, Time series API works well with [IoT Time Series Service](api-iottimeseries-api.html). For historical data older than 30 minutes and overall data size > 1 MB, it is recommended to use [IoT TS Bulk Services](../iot-iottsbulk/api-iottsbulk-api.html). - Time series Api limit is 2000 records for a data Query. If demand is higher than 2000 records involving data older than 4 hours, it is recommended to use the [IoT TS Bulk Read Services](../iot-iottsbulk/api-iottsbulk-api.html). - If results of some data queries are to be further utilized for Analytical purpose, IDL Timeseries Import can be leveraged instead of multiple calls of Time Series API. - Data query performance varies and can be on a higher side with increase in Asset modelling complexity (More number of variables, Big String data types, date range etc.) Modelling recommendations are available at [Asset Management Service - Modeling Recommendation](../advanced-assetmanagement/api-assetmanagement-overview.html). - Real/near-real time data should be of recent time-range to observe a low data query latency. Queries for older data are subjected to have higher latency. - It is recommended to use ‘GET’ Time series Queries with a specified time range for better throughput and performance. - Same, repetitive & similar time range data query should be avoided during continuous polling. i.e., Repetitive Queries for last few months data, sending every minute will hold build huge load on system with only value addition of only last 1 minute data over previous. Such type of redundant queries should be avoided as much as possible. - Longer time range should be split into smaller time ranges to experience faster query results. - If data of multiple variables needs to be queried, using a single api call instead of multiple calls is preferable. - For applications frequently querying multiple days of data, it is recommended to break the time range and do incremental time range based queries. - For Aggregate queries, consumer applications should provide an option to select Aggregate window (intervalValue & intervalUnit). - It is recommended to not try either of the operation (put/delete) together for given tenant/asset/aspect/timerange combination unless first operation is completed to avoid any data inconsistencies. ### Rate limit and retry Recommendation - It's best practice to include error handling for 429 responses in your code. The 429 status code means too many requests. The Retry-After header specifies no. of seconds after which API call can be retried. Your code should stop making additional API requests until enough time has passed to retry. #### Reducing the number of API requests - Optimize your code to eliminate any unnecessary API calls. For example, are some requests getting data items that aren't used in application? - Cache frequently used data. Application can cache data on the server or on the client. Application can also save relatively static information in a database. - Sideload related data. Application can avoid unnecessary API calls by sideloading one set of records with another. Sideloading lets you get two sets of records in a single request. For Example, If data for multiple variable of an aspect is requested in separate requests in parallel, it should be sideloaded to single request. - Use bulk endpoints where ever possible, that let you ingest/read data in single API request. However bulk endpoint may not be performant or sync endpoint, but may benefit in few scenarios to avoid throttling. ##### Regulating the request rate - If you regularly exceed the rate limit, update logic to distribute requests more evenly over a period of time. This is known as a throttling process or a throttling controller. Regulating the request rate can be done statically or dynamically. For example, monitor your request rate and regulate requests when the rate approaches the rate limit. ### For Optimizing Read Rate - [Asset modeling recommendations](https://documentation.mindsphere.io/MindSphere/apis/advanced-assetmanagement/api-assetmanagement-overview.html#modeling-recommendation) - Use "select" while data queries timeseries raw or aggregate data. This will reduce the response size as per requirement and hence reduce the Read/Compute requirements. For Aggregate select you may refer [this link](https://documentation.mindsphere.io/MindSphere/apis/iot-iottsaggregates/api-iottsaggregates-references-select.html) - Leverage [Timeseries subscription notification](https://documentation.mindsphere.io/MindSphere/apis/iot-iottimeseries/api-subscription-notification-overview.html) for new data arrival feature to get notified when to make data queries instead of continuous polling. - Application should be aware of throttling and should have an ability to handle such responses by either distributing calls over time or make their end users aware so that they do not make mistakes. ## Limitations - The maximum payload size for data upload for endpoint PUT/timeseries/{entityId}/{propertySetName} is 1 MB . - The Data Ingest rate throttling limit is 100 KB/s for Asset/Aspect combination in PUT Time series API. - For the new P&P environments, historical data Ingest allowed for past data is up to 366 days and for future data is up to 31 days. - Double can have up to 38 digits of precision,and can be positive, negative, or zero. - Positive range for Double : 1E-130 to 9.9999999999999999999999999999999999999E+125 - Negative range for Double : -9.9999999999999999999999999999999999999E+125 to -1E-130 - Requests are processed in order of arrival. Thus, the latest stored record might not reflect the latest request. - Data Ingest Timeseries precision is reduced to millisecond precision for Timeseries Storage, Query and further data related processing. For Example, any granular precision like nanosecond will be converted and stored in millisecond precision - Storage of empty strings is not supported. The respective fields are ignored. - Queries with the URL parameter `latestValue` are not available for String and Timestamp variables. - Queries with the URL parameter `latestValue` do not consider records ingested more than 3 months or less than ~5 minutes ago. - Queries with the URL parameter `latestValue` brings latest value for near real time data ingestion. If data ingestion is delayed and past data is ingested which is older than ~2 hours; it will have similar delays to fetch latest values. Note The 'latestValue' support for near real time data ingestion is not available for private cloud offerings as there is expected delay. It is recommended to use timeseries get without range with a 'select' parameter. - Queries without specified timestamps and older than 12 months will provide empty response. - Queries with 'limit' parameters works precisely in case 'select' is not used together. In case of 'select' with 'limit'; the response is further reduced by actual variables available in retrieved limit values. The default limit is 2000 timeseries records. - Queries with 'limit' parameters works precisely in case there is no empty records (empty records only contains timestamp without any value on any aspect variables) for requested range. In case it contains empty records; the response is further reduced by total number of empty records in retrieved limit values. The default limit is 2000 timeseries records. - After Asset Model update (i.e. updating asset, aspect,assetType, variables etc.), it may take up to 1 minute for internal sync and processing successful requests for timeseries data ingestion, reads. - Queries without To & From (which is [Get the latest record query](https://documentation.mindsphere.io/MindSphere/apis/iot-iottimeseries/api-iottimeseries-samples.html#get-the-latest-record)) work precisely in case of no empty records (empty records only contains timestamps without any value on any aspect variables). In case it contains empty records as latest data ingestion records; the response will be empty. - The maximum number of aspects (one or multiple assets) per data upload request is 5, with a maximum payload size of 100 KB and 100 records for endpoint PUT/timeseries. - The '+' character used to include timezone in timestamp must be encoded to '%2B' in time range. For example, 2017-09-21T14:08:00%2B02:00 instead of 2017-09-21T14:08:00+02:00. - Time series data will not be retrieved via GET endpoint of Time series services for Data Ingested via [IoT TS Bulk Services](../iot-iottsbulk/api-iottsbulk-api.html) for past 7 days. - For Time series Data ingestion of past 7 days, use either IoT Time series Services or [IoT TS Bulk Services](../iot-iottsbulk/api-iottsbulk-api.html). If both the methods are used, data consistency will be compromised. - If customer tries the operation (Data Ingest/Delete) together for given tenant/asset/aspect/timerange combination, there might be data inconsistencies. - For Private Cloud Deployments, - Timeseries will not accept data older than (T - 168) hours. - Timeseries will not accept future data (T + 0) hours. - Usage metrics in UTS is not available for Timeseries Storage. - For Timeseries Data Ingestion below special characters are not supported in STRING or BIGSTRING which contains '\\u0000'. To get the current list of known restrictions, go to [release notes](https://documentation.mindsphere.io/resources/html/release-notes/en-US/index.html) and choose the latest date. From there go to "MindAccess Developer Plan Subscribers and MindAccess Operator Plan Subscribers" and pick the IoT service you are interested in. ## Example Scenario A wind turbine is connected to Industrial IoT. The blades move with a constantly changing speed. A sensor measures the speed and sends the data via MindConnect to Insights Hub. The IoT Time Series Service API allows you to write and read the speed data of the wind wheel. ## Related Links - [Asset Management](../advanced-assetmanagement/api-assetmanagement-overview.html) - [IoT TS Aggregates Service](../iot-iottsaggregates/api-iottsaggregates-overview.html) # IoT Time Series Service – Samples As soon as your device is connected to Insights Hub and sends time series data, you will be able to gain insights by doing analysis and visualization. Time series data is always stored against an aspect of an asset. ## Prerequisites - You have created an aspect defining the types of data you want to collect. - You have assigned the aspect to an asset in order to be able to store time series data. ## Writing Time Series Data The samples in this section are based on the `PUT` endpoint for which [limitations](api-iottimeseries-overview.html#limitations) apply. Attention If data for an already existing timestamp is uploaded, the complete record for this timestamp including all variables is replaced. Variables which are not provided are set to NULL. ### Create a Single Time Series Record Call the following endpoint without additional URL parameters to create a time series record for an aspect: ```http https://gateway.{region}-{environment}.{mindsphere-domain}/api/iottimeseries/v3/timeseries/{assetId}/{aspectName} ``` The endpoint expects payload containing a timestamp filed `_time` and fields for the variables of the selected aspect. The timestamp must adhere with ISO 8601. Sample payload for an aspect with two numeric variables `temperature` and `pressure`, and a string variable `notes`: ```json [ { "_time": "2017-11-02T00:00:00.001Z", "temperature": 10, "pressure": 2, "notes": "normal operation" } ] ``` ### Create Multiple Time Series Records Create multiple records with different timestamps using a single request to the same endpoint as above. Sample Payload: ```json [ { "_time": "2017-11-02T00:00:00.001Z", "temperature": 10, "pressure": 2, "notes": "normal operation" }, { "_time": "2017-11-02T00:01:00.001Z", "temperature": 17, "pressure": 2, "notes": "normal operation" } ] ``` ### Create Time Series Records for Multiple Aspects Create time series data for multiple aspects of the same or different assets using the following endpoint: ```http https://gateway.{region}-{environment}.{mindsphere-domain}/api/iottimeseries/v3/timeseries; ``` Info All aspects must belong to the same environment. Sample Payload: ```json { "timeseries": [ { "entityId": "{assetId_1}", "propertysetName": "{aspectName_1}", "data": [ { "enginetemperature_1": 100, "speed_1": 100, "_time": "2019-03-25T01:00:00Z" }, { "enginetemperature_2": 200, "speed_2": 200, "_time": "2019-03-25T02:00:00Z" } ] }, { "entityId": "{assetId_2}", "propertysetName": "{aspectName_2}", "data": [ { "rpm_1": 10, "pressure_1": 200, "_time": "2019-03-25T01:00:00Z" } ] } ] } ``` Any data sent to the endpoint is asynchronously written into the underlying data store. Hence, there might be a delay before you can read back the data from the service. Validation of the payload is done synchronously so that the client receives immediate feedback. ## Reading Time Series Data The samples in this section are based on the `GET` endpoint for which [limitations](api-iottimeseries-overview.html#limitations) apply. A response from this endpoint contains up to 2,000 records. If more records are available, the response contains a `Link` header with a URI to get the remaining records, such as: ```http https://gateway.{region}-{environment}.{mindsphere-domain}/api/iottimeseries/v3/timeseries/{assetId}/{aspectName}; rel="next" ``` ### Get Data of a Specific Variable Specify a variable of an aspect using the URL parameter `select`: ```http https://gateway.{region}-{environment}.{mindsphere-domain}/api/iottimeseries/v3/timeseries/{assetId}/{aspectName}?select=temperature ``` The `select` parameter is case-insensitive. It accepts a comma separated list of variable names and quality code variable names, which suffix the respective variable names with `_qc`.\ The `select` parameter can be combined with the URL parameters described in the following sections. Info The select query excludes other variables from the response, but does not check if the specified variable is available. The response may contain time series records which only consist of the timestamp field. Sample Response: ```json [ { "temperature": 17, "_time": "2017-11-01T01:00:00.001Z" } ] ``` ### Get the Latest Record This endpoint is preferred when aspect data is changing frequently. Call the following endpoint without additional URL parameters to read the latest record of an aspect: ```http https://gateway.{region}-{environment}.{mindsphere-domain}/api/iottimeseries/v3/timeseries/{assetId}/{aspectName} ``` Latest Record vs. Latest Value Note that the latest record might only contain a timestamp. For reading the latest received values, use the [respective URL parameter](#get-the-latest-values). Sample Response: ```json [ { "temperature": 12, "pressure": 4, "notes": "normal operation", "_time": "2017-11-02T00:00:00.001Z" } ] ``` ### Get the Latest Values This endpoint is preferred when aspect data is changing non-frequently. Request the latest values using the URL parameter `latestValue`: ```http https://gateway.{region}-{environment}.{mindsphere-domain}/api/iottimeseries/v3/timeseries/{assetId}/{aspectName}?latestValue=true; ``` Latest Record vs. Latest Value Note that this request returns the latest received value and associated timestamp for each variable. For reading the latest received record, use the [respective request](#get-the-latest-record). Sample Response: ```json [ { "enginetemperature": 11234570, "speed": 3, "_time": "2019-05-15T04:30:00Z" }, { "rpm": 13.12, "_time": "2019-05-16T04:30:00Z" }, { "pressure": 4, "_time": "2019-05-16T05:30:00Z" } ] ``` ### Get Data from a Specific Time Range Specify the desired time range using the URL parameters `from` and `to`: ```http https://gateway.{region}-{environment}.{mindsphere-domain}/api/iottimeseries/v3/timeseries/{assetId}/{aspectName}?from=2017-11-01T00:00:00Z&to=2018-01-01T00:00:00Z ``` For setting the time zone of the time range, the `+` character must be encoded to `%2B` in the URL. For example, `2017-09-21T14:08:00%2B02:00` instead of `2017-09-21T14:08:00+02:00`. Sample Response: ```json [ { "temperature": 10, "pressure": 2, "notes": "normal operation", "_time": "2017-11-01T00:00:00.001Z" }, { "temperature": 17, "pressure": 2, "notes": "normal operation", "_time": "2017-11-01T01:00:00.001Z" } ] ``` Timestamps in the response are given in ISO8601 format in UTC. The service returns time series records with timestamps greater than the `from` time and less than or equal to the `to` time. The respective records in the response are sorted in ascending order, by default. The order can be changed using the optional URL parameter `sort` (currently only available in region Europe 1) which accepts either `desc` or `asc`. Bad Request If the endpoint responds with HTTP 400 Bad Request, it might be caused by: - The specified timespan is greater than 90 days. - The `from` field is greater than the `to` field. ### Get a Defined Number of Records after a Given Start Time Specify the desired number of records and the start time using the URL parameters `from` and `limit`: ```http GET https://gateway.{region}-{environment}.{mindsphere-domain}/api/iottimeseries/v3/timeseries/{assetId}/{aspectName}?from=2017-11-01T00:00:00Z&limit=4 ``` Sample Response: ```json [ { "temperature": 10, "pressure": 2, "notes": "normal operation", "_time": "2017-11-01T00:00:00.001Z" }, { "temperature": 17, "pressure": 2, "notes": "normal operation", "_time": "2017-11-01T01:00:00.001Z" } ] ``` If there are fewer results available than the defined `limit`, the response only contains the available records. ## Deleting Time Series Data The samples in this section are based on the `DELETE` endpoint for which [limitations](api-iottimeseries-overview.html#delete-limitations) is applicable. ### Valid Delete Request Examples Specify the desired hourly aligned time range using the URL parameters `from` and `to`: ```http DELETE https://gateway.{region}-{environment}.{mindsphere-domain}/api/iottimeseries/v3/timeseries/{assetId}/{aspectName}?from=2017-11-01T00:00:00Z&to=2017-11-01T01:00:00Z ``` The specified time range cannot be more than 7 days: ```http DELETE https://gateway.{region}-{environment}.{mindsphere-domain}/api/iottimeseries/v3/timeseries/{assetId}/{aspectName}?from=2017-11-01T00:00:00Z&to=2017-11-08T00:00:00Z ``` ### Invalid Delete Request Examples The specified time range must be aligned hourly. The following request would fail if the `from` and `to` is not aligned to the UTC hours: ```http DELETE https://gateway.{region}-{environment}.{mindsphere-domain}/api/iottimeseries/v3/timeseries/{assetId}/{aspectName}?from=2017-11-01T00:15:00Z&to=2017-11-01T02:15:00Z ``` The specified time range must be at most 7 days. The following request would fail if the duration between `from` and `to` is greater than 7 days: ```http DELETE https://gateway.{region}-{environment}.{mindsphere-domain}/api/iottimeseries/v3/timeseries/{assetId}/{aspectName}?from=2017-11-01T00:00:00Z&to=2017-12-01T00:00:00Z ``` # Notification for New Arrival of Timeseries Data ## Overview Currently, to fetch Time Series data, an Insights Hub application would have to query all asset instances, compare the timestamps, find the latest Time Series data and update the application. This would involve multiple requests through Industrial IoT Gateway which is expensive and not optimal or customer friendly. With subscription based notification, 3rd party applications will get notified by Insights Hub on new Time Series data arrival for concrete Assets and Aspect combination in a time range of 1 minute. This mechanism enables customer when to poll the Time series data. this improves the usability of Insights Hub Services and user experience for the developers. To create and publish App for consuming Timeseries data arrival notification, refer to [How to Subscribe to notifications from Insights Hub for new data arrival](https://documentation.mindsphere.io/MindSphere/howto/howto-receive-notifications.html) Note Subscription based notification is add-on offering on Public Cloud. ## Limitation - The underlying messaging service may bring nomenclature/version changes in near term, which may need changes and adaption in customer application. The subscription notification functionality provides notification on data arrival and not the actual timeseries data. - Notification is at granularity of environment level. Currently, granular notification at asset, aspect or asset attribute is not supported. - Only API type Mind-apps which are hosted behind Industrial IoT Gateway can access notifications. ## References - [Message Broker API](../core-messagebroker/api-messagebroker-api.html) - Provisioning API ## FAQs 1. What is notification level in asset model? - The notification will be available at environment level. We also provide aspect type information in the application payload. 1. What is the time interval of notification? - The time interval of the notification is 1 minute. 1. Can customer get delayed notification? - Yes, in case any Industrial IoT component is non-operational. # IoT Times Series Aggregates Service – API Overview Region Europe 1 New [Download OpenAPI Specification](/insights-hub/api_specs/iottsaggregates-v4-4-0-xcelerator.yaml) Old [Download OpenAPI Specification](/insights-hub/api_specs/iottsaggregates-v3.6.0.yaml) # IoT Time Series Aggregates Service – Request Parameters ## Interval Length ### Performance Entity Only pre-calculated interval length are supported. Choose an interval length that is an exact multiple of a pre-calculated aggregation interval, such as 10 minutes, 2 hours, 5 days, 3 weeks, or 2 months. ## Start Time and End Time Precision In case the parameters Start and End Time are provided but do not coincide with the pre-calculated interval boundaries of the used interval, Start and End Time are shifted such that the overall time range contains the provided one and time range boundaries coincide with interval boundaries. For example, a request is made with Start Time '2020-01-10T01:10:00Z', End Time '2020-01-10T02:50:00Z' and interval length '1 hour'. Here, Start and End time does not coincide with *hour* boundary, so it will be adjusted to nearby *hour* boundary, i.e, Start Time '2020-01-10T01Z' and End Time '2020-01-10T03Z'. ## Automatic Interval Length calculation If the interval length is not provided, the largest available interval length fitting into the used time range is chosen. For example, a request is made with Start Time `2020-01-10T01:10:00Z`, End Time `2020-01-10T01:20:00Z`. Here, interval length is automatically calculated using Start and End time provided. Since difference between Start and End time is less than an hour, interval length is calculated as `1 minute`. ## Parameter `count` ### Parameter `count`: Use Case 1 If the interval length and count are provided, it suffices to either provide Start Time or End Time in addition. The missing parameter is determined based on the time range computed from interval length and count. For example, a request is made with Start Time `2020-01-10T01:00:00Z`, interval length `1 hour` and count `4`. Here, 4 aggregates of 1 hour is returned between Start Time `2020-01-10T01:00:00Z` and End Time `2020-01-10T05:00:00Z`. ### Parameter `count`: Use Case 2 If from, to and count are provided, `intervalUnit` and `intervalValue` are determined based on the time range divided by `count`. For example, a request is made with Start Time `2020-01-10T01:00:00Z`, End Time `2020-01-10T02:00:00Z` and count `3`. Here, the difference between Start and End time is 24 hours and 3 aggregates response is requested, so interval length returned is `8 hours`. ## Parameter Selection There is an optional `select` parameter that defines which data to return. For more information, refer [IoT Time Series Aggregates Service-Selecting](api-iottsaggregates-references-select.html). # IoT Time Series Aggregates Service – Request Parameters ## Ideal Interval Length ### Performance Entity For best performance, choose an interval length which is an exact multiple of a pre-calculated aggregation interval, such as 10 minutes, 2 hours, 5 days, 3 weeks, or 2 months. Requests for other interval lengths take longer to process, because they require using raw time series and pre-calculated aggregations to calculate results. ### Simulation Entity Such entities support only the following intervals - 1ms,10ms and 1s. ## Start Time Precision The requested start and end times cannot be defined with higher precision than the requested interval length. The total time range must be a multiple of the requested interval length. For example, assume a request for data aggregated into intervals of 5 minutes. There, the start time must be defined with minute precision and the total time range must be a multiple of 5 minutes. When requesting data aggregated into intervals of 10 seconds, the start time must be set with second precision and the total time range must be a multiple of 10 seconds. ## Interval Precision The supported interval lengths overlap, which provides flexibility to the application. For example, 1 hour and 60 minutes represent the same length, but allow different behavior. A request for aggregation intervals of 1 hour can only be used for time ranges which start and end at the top of the hour, like 5:00 – 10:00. In contrast, a request for aggregation intervals of 60 minutes can also be used for time ranges like 5:30 – 10:30, which would otherwise be rejected. Accordingly, if the environment is configured with the first day of the week being Monday, a time range of 1 week must start and end on a Monday. A time range of 7 days could start on any day of the week. For more information refer to the Samples. ## Parameter Selection There is an optional `select` parameter that defines which data to return. For more information, refer to the [Selecting](api-iottsaggregates-references-select.html) documentation. # IoT Time Series Aggregates Service ## Idea The IoT Time Series (TS) Aggregates Service creates aggregated summaries of numeric time series data and provides interfaces to read them. This allows applications to retrieve smaller data sets that cover a long time range with much better performance than processing all the raw time series data. For example, an aspect could create new data every second, which adds up to ~2.5 million records per month. An application could use the IoT TS Aggregates Service to a summary for each day of the month, obtaining only 30 records. ## Access For accessing this service, you need to have the respective roles listed in [IoT Time Series Aggregates Service roles and scopes](https://documentation.mindsphere.io/MindSphere/concepts/concept-roles-scopes.html#iot-time-series-aggregates). A user can only read data within their environment and subtenants. ## Basics ### Aggregated Data The data summarized in an aggregate is the data with timestamps greater than the start time and less than or equal to the end time of the interval. This ensures that count type data is correctly reflected in the aggregate. For example, if a device provides data at the end of every five minutes, the hourly aggregation includes the data with timestamps from five minutes after the hour until the end of the hour (e.g., 3:05 to 4:00). Each aggregate summarizes the following information: - value and time of the first data point of the interval - value and time of the last data point of the interval - value and time of the minimum data point during the interval - value and time of the maximum data point during the interval - the sum of the values during the interval - the average of the values during the interval - the count of the good, uncertain, and bad values during the interval - standard deviation of the values within the interval ### Pre-Calculated Aggregates The IoT TS Aggregates Service automatically creates pre-calculated aggregates and stores them for later retrieval to enhance performance of aggregate queries. Aggregates of the following interval lengths are pre-calculated: | Interval Length | Description | | --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ | | 1 minute | Aggregates end at 1 minute after the hour, 2 minutes after the hour, and so on. They are created within two minutes after the one-minute interval has ended. | | 1 hour | Aggregates end at the top of the hour. They are created within eight minutes after the end of the hour. | | 1 day | Aggregates end at midnight in the asset's time zone. They can take up to 1 day post data ingestion. | The aggregated data is wall clock aligned based on the asset's time zone. Note Aggregated data is not immediately available. For example, if aggregated data of a performance asset is requested at 10:00 with interval length of 1 hour, the response will not contain the aggregate from 9:00 to 10:00, since it has not been calculated yet. ### On Demand Aggregates On demand aggregates are available on top of Pre-Calculated aggregates. They are created when a client requests aggregated data, which cannot be calculated using pre-calculated aggregates (e.g. interval length of 4 minute or starting at 10:00). The IoT TS Aggregates Service uses suitable pre-calculated aggregates and generates aggregates on demand.\ The following interval lengths are supported: - 1-60 minutes - 1-24 hours - 1+ days - 1+ weeks - 1+ months ### Aggregate Creation #### Data Availability On performance assets, aggregated data is only created if time series data exists for that time range. Aggregates are created based on the data that is available at the time. Data that arrives late is not included in the initial aggregated results, but triggers a re-calculation after a delay. #### Quality Code Data If a variable is configured to contain quality code data, the service uses it to rate the aggregates. The quality code is compared to the good and uncertain thresholds defined in the environment. By default, the values are set according to the OPC UA standard. Good data is defined as anything less than or equal to the `goodThreshold` value.\ Uncertain data is defined as anything greater than the `goodThreshold` value and less than or equal to the `uncertainThreshold` value.\ Bad data is defined as anything greater than the `uncertainThreshold`. Only good and uncertain data is used to calculate the summary values listed above. Bad data is ignored for these calculations. Each aggregate contains counts of the three types of data: `countgood`, `countuncertain`, and `countbad`. If no quality code data is available for a time series record, it is considered good. ## Features The IoT TS Aggregates Service exposes its API for realizing the following tasks: - Read aggregated interval data using pre-calculated aggregates ## Usage Recommendations - For Aggregate queries, consumer applications should provide an option to select Aggregate window (`intervalValue` and `intervalUnit`). ## Usage Quota and Limits The usage quota and limits details for the IoT TS services is mentioned in [API rate limits](../iot-iottimeseries/api-iottimeseries-apiratelimits.html). ## Limitations - Only pre-calculated interval length are supported in aggregate query. - Querying aggregates for Simulation asset is not supported. - Maximum date range for querying 'minute' aggregate is 48 hours. - Maximum date range for querying 'hour' aggregate is 31 days. - Maximum date range for querying 'day/week/month' aggregate is 366 days. - Aggregates are not supported for variables with data type "STRING", "BIG_STRING", "TIMESTAMP". - Daily Aggregates are available for the current day, representing aggregate data for ongoing data ingestion. However, during midday, daily aggregates for the current day may have inconsistencies since they will be updated approximately every 1 hour till the end of the day. If customer needs near to consistent result for Daily Aggregate; they may discard current day aggregates until day is over. - Time zones that are 15 or 45 minutes off a UTC hour (e.g. Nepal standard time (UTC+05:45)) are not supported. Time zones that are an integer number of hours off GMT or 30 minutes off a UTC hour (e.g. India (UTC+05:30)), are supported. - If all 4 parameters are provided (from, to, intervalUnit, intervalValue) - "from" parameter will be adjusted depending on the "to" parameter and the multiplication factor of "intervalUnit" and "intervalValue" - Count has specific behavior's when used beyond mentioned scenario in API. - If all 5 parameters are provided (from, to, intervalUnit, intervalValue, and count) - Count will be used to re-calculate "from" parameter and override requested "from" with count of interval for providing response - Re-calculated "from" = "To" - ["Count" * ("intervalvalue"+"intervalunit")] - Count parameter will be used to fetch no. of sequential aggregates. If there is blank aggregate in sequence, response will be reduced to actual present aggregates and may differ from requested aggregate "count" To get the current list of restrictions, navigate to the [Release Notes](https://documentation.mindsphere.io/resources/html/release-notes/en-US/index.html), choose the latest date and refer IoT & Storage Services: Aggregates Service. ## Example Scenario A wind turbine produces time series data for the speed of the blades. The data is sent continuously to Insights Hub. The IoT TS Aggregates Service API allows you to collect aggregated data for the speed of the windmill. ## Related Links - [Asset Management Service](../advanced-assetmanagement/api-assetmanagement-overview.html) - [IoT Time Series Service](../iot-iottimeseries/api-iottimeseries-overview.html) # IoT Time Series Aggregates Service Announcement Aggregate V4, the new API is now available since June 2020. For more information about new features and limitations, refer [Version 4.0 - What's New](api-iottsaggregates-version-v4-whats-new.html). Older version of Aggregate V3 will be available without any support and reduced quota till 30th June'2024. For more information, refer [Migration guide](api-iottsaggregates-v3-v4-migration-guide.html). From 1st March 2022 onwards, new tenants or tenants migrated from older offerings to new packaging will be provisioned only with Aggregate V4 API. ## Idea The IoT Time Series (TS) Aggregates Service creates aggregated summaries of numeric time series data and provides interfaces to read them. This allows applications to retrieve smaller data sets that cover a long time range with much better performance than processing all the raw time series data. For example, an aspect could create new data every second, which adds up to ~2.5 million records per month. An application could use the IoT TS Aggregates Service to a summary for each day of the month, obtaining only 30 records. ## Access For accessing this service, you need to have the respective roles listed in [IoT Time Series Aggregates Service roles and scopes](https://documentation.mindsphere.io/MindSphere/concepts/concept-roles-scopes.html#iot-time-series-aggregates). A user can only read data within their environment and subtenants. For accessing the Secure Data Sharing (SDS) protected APIs, you need to have appropriate Policy Definitions in place. For the list of supported APIs and required actions, refer to the [Required Policy actions by Industrial IoT services](../core-resourceaccessmanagement/api-resourceaccessmanagement-services-actions-map.html#iot-time-series-service) chapter. ## Basics Hint In the IoT context, assets are referred to as `entity` and aspects as `propertyset`. ### Aggregated Data The data summarized in an aggregate is the data with timestamps greater than the start time and less than or equal to the end time of the interval. This ensures that count type data is correctly reflected in the aggregate. For example, if a device provides data at the end of every five minutes, the hourly aggregation includes the data with timestamps from five minutes after the hour until the end of the hour (e.g., 3:05 to 4:00). Each aggregate summarizes the following information: - value and time of the first data point of the interval - value and time of the last data point of the interval - value and time of the minimum data point during the interval - value and time of the maximum data point during the interval - the count of data points during the interval - the sum of the values during the interval - the average of the values during the interval - the count of the good, uncertain, and bad values during the interval - standard deviation of the values within the interval ### Pre-Calculated Aggregates The IoT TS Aggregates Service automatically creates pre-calculated aggregates and stores them for later retrieval to enhance performance of aggregate queries. Aggregates of the following interval lengths are pre-calculated: | Interval Length | Description | | --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 1 millisecond | Aggregates end at 1 millisecond after the hour, 2 milliseconds after the hour, and so on. *This interval length is only available on simulation entities.* | | 10 milliseconds | Aggregates end at 10 milliseconds after the hour, 20 milliseconds after the hour, and so on. *This interval length is only available on simulation entities.* | | 1 second | Aggregates end at 1 second after the hour, 2 seconds after the hour, and so on. *This interval length is only available on simulation entities.* | | 1 minute | Aggregates end at 1 minutes after the hour, 2 minutes after the hour, and so on. They are created within two minutes after the one-minute interval has ended. *This interval length is only available on performance entities.* | | 1 hour | Aggregates end at the top of the hour. They are created within eight minutes after the end of the hour. | | 1 day | Aggregates end at midnight in the asset's time zone. They are created within fifteen minutes after midnight in UTC time. | The aggregated data is wall clock aligned based on the asset's time zone. Note Aggregated data is not immediately available. For example, if aggregated data of a performance entity is requested at 10:00 with interval length of 1 hour, the response will not contain the aggregate from 9:00 to 10:00, since it has not been calculated yet. ### On Demand Aggregates On demand aggregates are only available on performance entities. They are created when a client requests aggregated data, which cannot be calculated using pre-calculated aggregates (e.g. interval length of 1 minute or starting at 10:01). The IoT TS Aggregates Service uses suitable pre-calculated aggregates and calculates the missing items on demand.\ The following interval lengths are supported: - 1-120 seconds - 1-60 minutes - 1-24 hours - 1+ days - 1+ weeks - 1+ months ### Aggregate Creation #### Data Availability On performance entities, aggregated data is only created if time series data exists for that time range. Aggregates are created based on the data that is available at the time. Data that arrives late is not included in the initial aggregated results, but triggers a re-calculation after a delay. On simulation entities, aggregated data is created when the ingestion job for time series data is completed. If data for the same hour is ingested by another job, the respective aggregates are re-calculated. #### Quality Code Data If a variable is configured to contain quality code data, the service uses it to rate the aggregates. The quality code is compared to the good and uncertain thresholds defined in the environment. By default, the values are set according to the OPC UA standard. Good data is defined as anything less than or equal to the `goodThreshold` value.\ Uncertain data is defined as anything greater than the `goodThreshold` value and less than or equal to the `uncertainThreshold` value.\ Bad data is defined as anything greater than the `uncertainThreshold`. Only good and uncertain data is used to calculate the summary values listed above. Bad data is ignored for these calculations. Each aggregate contains counts of the three types of data: `countgood`, `countuncertain`, and `countbad`. If no quality code data is available for a time series record, it is considered good. ## Features The IoT TS Aggregates Service exposes its API for realizing the following tasks: - Read aggregated interval data using pre-calculated aggregates - Generate aggregated interval data for custom intervals ## Limitations - On demand aggregates can only be created from up to 5,000 raw time series entries. - On demand aggregates are only available for performance entities. - On the fly aggregate query response is slower than pre-calculated aggregate request. - Aggregates are not supported for variables with data type "STRING", "BIG_STRING", "TIMESTAMP". - A query can only return up to 200 aggregate intervals in one response. - Time zones that are 15 or 45 minutes off a UTC hour (e.g. Nepal standard time (UTC+05:45)) are not supported. Time zones that are an integer number of hours off GMT or 30 minutes off a UTC hour (e.g. India (UTC+05:30)), are supported. - The '+' character used to include timezone in timestamp must be encoded to '%2B' in time range. For example, 2017-09-21T14:08:00%2B02:00 instead of 2017-09-21T14:08:00+02:00. To get the current list of restrictions, navigate to the [release notes](https://documentation.mindsphere.io/resources/html/release-notes/en-US/index.html), choose the latest date and refer the IoT TS Aggregates Service. ## Example Scenario A wind turbine produces time series data for the speed of the blades. The data is sent continuously to Industrial IoT. The IoT TS Aggregates Service API allows you to collect aggregated data for the speed of the windmill. ## Related Links - [Asset Management Service](../advanced-assetmanagement/api-assetmanagement-overview.html) - [IoT Time Series Service](../iot-iottimeseries/api-iottimeseries-overview.html) # IoT Time Series Aggregates Service – Selecting ## Select Syntax With the optional `select` parameter, it is possible to define which data to return. The value of the `select` parameter is a comma separated list of requested data. Each value in the list can either be a variable name or an aggregated field using a `variableName.fieldName` syntax. The field name can be one of the following: - `countgood` - `countuncertain` - `countbad` - `sum` - `average` - `mintime` - `minvalue` - `maxtime` - `maxvalue` - `firsttime` - `firstvalue` - `lasttime` - `lastvalue` - `sd` If no `select` parameter is specified, all variables and fields are returned. If a variable is specified, but no field name, all the fields for that variable are returned. If a variable and field name are specified, only that field of the variable is returned. Variable names are case-insensitive. Note For charting, dashboarding customer should use `select` for critical fields to be plotted like `average`, `minvalue`, `maxvalue`. This will reduce read size drastically. It reduces read rate in tune of 50-70% basis on specific use case. # IoT Time Series Aggregates Service – Samples for v4.x ## Requesting Aggregated Data ### Performance Entity In this example, there is an asset type `forklift` with an aspect `tireMonitor` and with the variables `pressure`, `temperature`, and `treadDepth`. The `pressure` variable has a quality code value of `Y` while the others are `N`. An instance of `forklift` with an asset ID of `978528e7a124458f87c8f1d38fd9400f` is defined. The following call requests data aggregated into 4 minute intervals from `2017-05-01T00:08:00Z` until `2017-05-01T00:16:00Z` for the `tireMonitor` aspect: ```http https://gateway.{region}-{environment}.{mindsphere-domain}/api/iottsaggregates/v4/aggregates?assetId=978528e7a124458f87c8f1d38fd9400f&aspectName=tireMonitor&from=2017-05-01T00:08:00Z&to=2017-05-01T00:16:00Z&select=pressure,temperature&intervalUnit=minute&intervalValue=4 ``` The following response is returned for this request: Response ```json { "aggregates": [ { "pressure": { "firsttime": "2017-05-01T00:09:00Z", "average": 93.75, "lasttime": "2017-05-01T00:12:00Z", "maxvalue": 95, "firstvalue": 93, "mintime": "2017-05-01T00:10:00Z", "lastvalue": 94, "countgood": 4, "countuncertain": 0, "countbad": 0, "sum": 375, "minvalue": 93, "maxtime": "2017-05-01T00:11:00Z", "sd": 14.36 }, "temperature": { "firsttime": "2017-05-01T00:09:00Z", "average": 44.25, "lasttime": "2017-05-01T00:12:00Z", "maxvalue": 45, "firstvalue": 43, "mintime": "2017-05-01T00:09:00Z", "lastvalue": 44, "countgood": 4, "countuncertain": 0, "countbad": 0, "sum": 177, "minvalue": 43, "maxtime": "2017-05-01T00:11:00Z", "sd": 14.36 }, "starttime": "2017-05-01T00:08:00Z", "endtime": "2017-05-01T00:12:00Z" }, { "pressure": { "firsttime": "2017-05-01T00:13:00Z", "average": 95, "lasttime": "2017-05-01T00:16:00Z", "maxvalue": 96, "firstvalue": 95, "mintime": "2017-05-01T00:15:00Z", "lastvalue": 96, "countgood": 4, "countuncertain": 0, "countbad": 0, "sum": 380, "minvalue": 94, "maxtime": "2017-05-01T00:16:00Z", "sd": 14.36 }, "temperature": { "firsttime": "2017-05-01T00:13:00Z", "average": 44.5, "lasttime": "2017-05-01T00:16:00Z", "maxvalue": 45, "firstvalue": 44, "mintime": "2017-05-01T00:14:00Z", "lastvalue": 44, "countgood": 4, "countuncertain": 0, "countbad": 0, "sum": 178, "minvalue": 43, "maxtime": "2017-05-01T00:15:00Z", "sd": 14.36 }, "starttime": "2017-05-01T00:12:00Z", "endtime": "2017-05-01T00:16:00Z" } ] } ``` ### Get Aggregate Data for a Specific Time Range with Select (variableName.fieldName) for Performance Entity Specify the desired time range using the URL parameters from and to along with select average value: ```http https://gateway.{region}-{environment}.{mindsphere-domain}/api/iottsaggregates/v4/aggregates?assetId=978528e7a124458f87c8f1d38fd9400f&aspectName=tireMonitor&from=2017-05-01T00:08:00Z&to=2017-05-01T00:16:00Z&intervalUnit=minute&intervalValue=4&select=pressure.average,temperature.average ``` The following response is returned for this request Response ```json { "aggregates": [ { "pressure": { "average": 93.75 }, "temperature": { "average": 44.25 }, "starttime": "2017-05-01T00:08:00Z", "endtime": "2017-05-01T00:12:00Z" }, { "pressure": { "average": 95 }, "temperature": { "average": 44.5 }, "starttime": "2017-05-01T00:12:00Z", "endtime": "2017-05-01T00:16:00Z" } ] } ``` Using `Select` will reduce the response size as per requirement and hence reduce the Read/Compute rate. ## Defining Time Range and Interval Length The following table lists the combinations of start time, end time, and interval length and explains the expected response: | Start Time | End Time | Interval Unit | Interval Value | Count | Explanation | | -------------------- | -------------------- | ------------- | -------------- | ----- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 2019-02-01Z | | day | 1 | 7 | End time calculated using Interval Unit, Interval Value and count. End Time = (2019-02-01Z) + (1 day) * 7 = (2019-02-08Z) | | 2019-02-01T10:00:00Z | | hour | 1 | | Current system time is '2019-02-01T20:00:00Z', then End time calculated as End Time = 2019-02-01T20:00:00 and count = (End time - Start time) / Interval Length, i.e. (2019-02-01T20:00:00Z - 2019-02-01T10:00:00Z) / 1 hour = 10 | | 2020-10-10T10:00:00Z | 2020-10-10T10:30:00Z | | | | Since the difference between Start Time and End time is less than 1 hour, Interval Length is calculated as '1 minute' with count = 30 | | 2020-10-10T10:00:00Z | 2020-10-10T12:00:00Z | | | | Since the difference between Start Time and End time is more than 1 hour and less than 1 day, Interval Length is calculated as '1 hour' with count = 2 | | 2020-10-10T00:00:00Z | 2020-10-15T00:00:00Z | | | | Since the difference between Start Time and End time is more than 1 day, Interval Length is calculated as '1 day' with count = 5 | | 2020-10-10T10:00:00Z | 2020-10-10T10:30:00Z | | | 3 | Since the difference between from and to date is 30 minute, and count asked is 3, Interval Length is calculated as '10 minute' | | 2020-10-10T02:10:00Z | 2020-10-10T09:10:00Z | hour | 1 | | Since aggregate is queried for 1 hour and dates are not aligned to Hour boundary, dates are adjusted to Hour boundary From = 2020-10-10T02:00:00Z and to = 2020-10-10T10:00:00Z | | 2020-10-10T00:00:00Z | 2020-10-11T00:00:00Z | hour | 4 | | Since pre-computed aggregate exists for every HOUR, Hourly aggregates will be combined and a response bucket every 4 hours will be returned | | 2020-10-11T04:00:00Z | 2020-10-11T07:00:00Z | hour | 4 | | Since pre-computed aggregate exists for every HOUR, Hourly aggregates will be combined and a response bucket of 4 hours will be returned with FROM = 2020-10-11T03:00:00Z and TO = 2020-10-11T07:00:00Z | | 2020-10-11T04:00:00Z | 2020-10-11T07:00:00Z | hour | 3 | | Since pre-computed aggregate exists for every HOUR, Hourly aggregates will be combined and a response bucket of 3 hours will be returned with FROM = 2020-10-11T04:00:00Z and TO = 2020-10-11T07:00:00Z | ## Get Day Aggregates with Date Range, timezone and Daylight Saving Time (DST) Day aggregates are stored in asset's timezone. Note Daylight Saving Time (DST) has no impact on Pre-Calculated 1 Hour and 1 Minute aggregate as they are created in UTC timezone.\* ### Query when DST is OFF If asset timezone is EST then day aggregates should be queried as mentioned below: ```http https://gateway.{region}-{environment}.{mindsphere-domain}/api/iottsaggregates/v4/aggregates/{assetId}/{aspectName}?from=2019-01-10T19:00:00Z&to=2019-01-12T19:00:00Z&intervalUnit=day&intervalValue=1 ``` Since EST timezone is `UTC-05:00`, for querying day aggregates hour part should be mentioned as `19:00:00`. The same query can be done like this, specifying timezone information in query: ```http https://gateway.{region}-{environment}.{mindsphere-domain}/api/iottsaggregates/v4/aggregates/{assetId}/{aspectName}?from=2019-01-11T00:00:00-05:00&to=2019-01-13T00:00:00-05:00&intervalUnit=day&intervalValue=1 ``` If a proper hour portion is not provided, then the service will automatically align dates to nearest Day boundary. For example: ```http https://gateway.{region}-{environment}.{mindsphere-domain}/api/iottsaggregates/v4/aggregates/{assetId}/{aspectName}?from=2019-01-10T00:00:00Z&to=2019-01-12T00:00:00Z&intervalUnit=day&intervalValue=1 ``` Since asset's timezone is EST, and timestamp is not aligned in query, the above query will be automatically aligned like the following: ```http https://gateway.{region}-{environment}.{mindsphere-domain}/api/iottsaggregates/v4/aggregates/{assetId}/{aspectName}?from=2019-01-09T19:00:00Z&to=2019-01-12T19:00:00Z&intervalUnit=day&intervalValue=1 ``` ### Query when DST is ON When querying a Date Range where DST is **ON**: ```http https://gateway.{region}-{environment}.{mindsphere-domain}/api/iottsaggregates/v4/aggregates/{assetId}/{aspectName}?from=2019-10-10T19:00:00Z&to=2019-10-12T19:00:00Z&intervalUnit=day&intervalValue=1 ``` Since day light is ON in October month, EDT timezone is `UTC-05:00`, day aggregates hour part should be mentioned as `19:00:00`. Same query can be done like this, specifying timezone information in query: ```http https://gateway.{region}-{environment}.{mindsphere-domain}/api/iottsaggregates/v4/aggregates/{assetId}/{aspectName}?from=2019-01-11T00:00:00-05:00&to=2019-01-13T00:00:00-05:00&intervalUnit=day&intervalValue=1 ``` If proper hour portion is not provided, the service will automatically align dates to nearest Day boundary. For example: ```http https://gateway.{region}-{environment}.{mindsphere-domain}/api/iottsaggregates/v4/aggregates/{assetId}/{aspectName}?from=2019-10-10T00:00:00Z&to=2019-10-12T00:00:00Z&intervalUnit=day&intervalValue=1 ``` Since asset's timezone is EST (DST is **ON**) and timestamp is not aligned in query, the above query will be automatically aligned to this: ```http https://gateway.{region}-{environment}.{mindsphere-domain}/api/iottsaggregates/v4/aggregates/{assetId}/{aspectName}?from=2019-10-09T19:00:00Z&to=2019-10-12T19:00:00Z&intervalUnit=day&intervalValue=1 ``` ### Query when DST is ON for start date and OFF for end date When querying a Date Range where start date falls when DST is **ON** and end date falls when DST if **OFF** ```http https://gateway.{region}-{environment}.{mindsphere-domain}/api/iottsaggregates/v3/aggregates/{assetId}/{aspectName}?from=2019-10-10T19:00:00Z&to=2019-11-10T20:00:00Z&intervalUnit=day&intervalValue=1 ``` Since day light is **ON** in October month, EDT timezone is `UTC-05:00`, day aggregates hour part should be mentioned as `19:00:00` for Start Date. Since day light is **OFF** in November month, EST timezone is `UTC-05:00`, day aggregates hour part should be mentioned as `19:00:00` for End Date. The same query can be done like this, specifying timezone information in query: ```http https://gateway.{region}-{environment}.{mindsphere-domain}/api/iottsaggregates/v3/aggregates/{assetId}/{aspectName}?from=2019-10-11T00:00:00-05:00&to=2019-11-11T00:00:00-05:00&intervalUnit=day&intervalValue=1 ``` If proper hour portion is not provided, the service will automatically align dates to nearest Day boundary. For example: ```http https://gateway.{region}-{environment}.{mindsphere-domain}/api/iottsaggregates/v3/aggregates/{assetId}/{aspectName}?from=2019-10-10T00:00:00Z&to=2019-11-10T00:00:00Z&intervalUnit=day&intervalValue=1 ``` Since asset's timezone is `EST` (Start time is in DST and End time is outside DST) and timestamp is not aligned in query, above query will be automatically aligned to this: ```http https://gateway.{region}-{environment}.{mindsphere-domain}/api/iottsaggregates/v3/aggregates/{assetId}/{aspectName}?from=2019-10-09T19:00:00Z&to=2019-11-10T19:00:00Z&intervalUnit=day&intervalValue=1 ``` # IoT Time Series Aggregates Service – Samples ## Requesting Aggregated Data ### Performance Entity In this example, there is an asset type `forklift` with an aspect `tireMonitor` and with the variables `pressure`, `temperature`, and `treadDepth`. The `pressure` variable has a quality code value of `Y` while the others are `N`. An instance of `forklift` with an asset ID of `978528e7a124458f87c8f1d38fd9400f` is defined. The following call requests data aggregated into 4 minute intervals from `2017-05-01T00:08:00` until `2017-05-01T00:16:00Z` for the `tireMonitor` aspect: ```http https://gateway.{region}-{environment}.{mindsphere-domain}/api/iottsaggregates/v3/aggregates/978528e7a124458f87c8f1d38fd9400f/tireMonitor?from=2017-05-01T00:08:00Z&to=2017-05-01T00:16:00Z&select=pressure,temperature&intervalUnit=minute&intervalValue=4 ``` The following response is returned for this request: Response ```json [ { "pressure": { "firsttime": "2017-05-01T00:09:00Z", "average": 93.75, "lasttime": "2017-05-01T00:12:00Z", "maxvalue": 95, "firstvalue": 93, "mintime": "2017-05-01T00:10:00Z", "lastvalue": 94, "countgood": 4, "countuncertain": 0, "countbad": 0, "sum": 375, "minvalue": 93, "maxtime": "2017-05-01T00:11:00Z", "sd": 14.36 }, "temperature": { "firsttime": "2017-05-01T00:09:00Z", "average": 44.25, "lasttime": "2017-05-01T00:12:00Z", "maxvalue": 45, "firstvalue": 43, "mintime": "2017-05-01T00:09:00Z", "lastvalue": 44, "countgood": 4, "countuncertain": 0, "countbad": 0, "sum": 177, "minvalue": 43, "maxtime": "2017-05-01T00:11:00Z", "sd": 14.36 }, "starttime": "2017-05-01T00:08:00Z", "endtime": "2017-05-01T00:12:00Z" }, { "pressure": { "firsttime": "2017-05-01T00:13:00Z", "average": 95, "lasttime": "2017-05-01T00:16:00Z", "maxvalue": 96, "firstvalue": 95, "mintime": "2017-05-01T00:15:00Z", "lastvalue": 96, "countgood": 4, "countuncertain": 0, "countbad": 0, "sum": 380, "minvalue": 94, "maxtime": "2017-05-01T00:16:00Z", "sd": 14.36 }, "temperature": { "firsttime": "2017-05-01T00:13:00Z", "average": 44.5, "lasttime": "2017-05-01T00:16:00Z", "maxvalue": 45, "firstvalue": 44, "mintime": "2017-05-01T00:14:00Z", "lastvalue": 44, "countgood": 4, "countuncertain": 0, "countbad": 0, "sum": 178, "minvalue": 43, "maxtime": "2017-05-01T00:15:00Z", "sd": 14.36 }, "starttime": "2017-05-01T00:12:00Z", "endtime": "2017-05-01T00:16:00Z" } ] ``` ### Get Aggregate Data for a Specific Time Range with Select (variableName.fieldName) for Performance Entity Specify the desired time range using the URL parameters from and to along with select average value: ```http https://gateway.{region}-{environment}.{mindsphere-domain}/api/iottsaggregates/v3/aggregates/{{entityId}}/{{propset}}?from=2017-05-01T00:08:00Z&to=2017-05-01T00:16:00Z&intervalUnit=minute&intervalValue=4&select=pressure.average,temperature.average ``` The following response is returned for this request: ```json [ { "pressure": { "average": 93.75 }, "temperature": { "average": 44.25 }, "starttime": "2017-05-01T00:08:00Z", "endtime": "2017-05-01T00:12:00Z" }, { "pressure": { "average": 95 }, "temperature": { "average": 44.5 }, "starttime": "2017-05-01T00:12:00Z", "endtime": "2017-05-01T00:16:00Z" } ] ``` Using `Select` will reduce the response size as per requirement and hence reduce the Read/Compute rate. ### Get Aggregate Data for a Specific Time Range with Time Zone for Performance Entity Specify the desired time range using the URL parameters from and to: ```http https://gateway.{region}-{environment}.{mindsphere-domain}/api/iottsaggregates/v3/aggregates/{assetId}/{aspectName}?from=2017-05-01T00:08:00%2B02:00&to=2017-05-01T00:16:00%2B02:00&intervalUnit=minute&intervalValue=4 ``` For setting the time zone of the time range, the `+` character must be encoded to `%2B` in the URL. For example, `2017-05-01T00:08:00%2B02:00` instead of `2017-05-01T00:08:00+02:00`. ### Get Day Aggregates with Date Range, timezone and Daylight Saving Time (DST) Day aggregates are stored in asset's timezone. Note Daylight Saving Time (DST) has no impact on Pre-Calculated 1 Hour and 1 Minute aggregate as they are created in UTC timezone. #### Query when DST is OFF If asset timezone is EST, then day aggregates should be queried as specified below: ```http https://gateway.{region}-{environment}.{mindsphere-domain}/api/iottsaggregates/v3/aggregates/{assetId}/{aspectName}?from=2019-01-10T20:00:00Z&to=2019-01-12T20:00:00Z&intervalUnit=day&intervalValue=1 ``` Since EST timezone is UTC-04:00, for querying day aggregates, the hour part should be mentioned as 20:00:00. Same query can be done as mentioned below, by specifying timezone information in query. ```http https://gateway.{region}-{environment}.{mindsphere-domain}/api/iottsaggregates/v3/aggregates/{assetId}/{aspectName}?from=2019-01-11T00:00:00-04:00&to=2019-01-13T00:00:00-04:00&intervalUnit=day&intervalValue=1 ``` #### Query when DST is ON When querying, a Date Range where DST is ON: ```http https://gateway.{region}-{environment}.{mindsphere-domain}/api/iottsaggregates/v3/aggregates/{assetId}/{aspectName}?from=2019-10-10T19:00:00Z&to=2019-10-12T19:00:00Z&intervalUnit=day&intervalValue=1 ``` Since day light is ON in October month, EDT timezone is UTC-05:00, day aggregates hour part should be mentioned as 19:00:00. Same query can be done as mentioned below by specifying timezone information in query: ```http https://gateway.{region}-{environment}.{mindsphere-domain}/api/iottsaggregates/v3/aggregates/{assetId}/{aspectName}?from=2019-01-11T00:00:00-05:00&to=2019-01-13T00:00:00-05:00&intervalUnit=day&intervalValue=1 ``` #### Query when DST is ON for start date and OFF for end date When querying a Date Range where start date falls when DST is ON and end date falls when DST if OFF, ```http https://gateway.{region}-{environment}.{mindsphere-domain}/api/iottsaggregates/v3/aggregates/{assetId}/{aspectName}?from=2019-10-10T19:00:00Z&to=2019-11-10T20:00:00Z&intervalUnit=day&intervalValue=1 ``` Since day light is ON in October month, EDT timezone is UTC-05:00, day aggregates hour part should be mentioned as 19:00:00 for Start Date. Since day light is OFF in November month, EST timezone is UTC-04:00, day aggregates hour part should be mentioned as 20:00:00 for End Date. Same query can be done as mentioned below, by specifying timezone information in the query. ```http https://gateway.{region}-{environment}.{mindsphere-domain}/api/iottsaggregates/v3/aggregates/{assetId}/{aspectName}?from=2019-10-11T00:00:00-05:00&to=2019-11-11T00:00:00-04:00&intervalUnit=day&intervalValue=1 ``` ### Simulation Entity The following call requests data aggregated into 1 millisecond intervals from `2017-05-01T00:08:00.001` until `2017-05-01T00:08:00.003Z` for the `tireMonitor` aspect: ```http https://gateway.{region}-{environment}.{mindsphere-domain}/api/iottsaggregates/v3/aggregates/978528e7a124458f87c8f1d38fd9400f/tireMonitor?from=2017-05-01T00:08:00.001Z&to=2017-05-01T00:08:00.003Z&select=pressure,temperature&intervalUnit=millisecond&intervalValue=1 ``` The following response is returned for this request: Response ```json [ { "pressure": { "firsttime": "2017-05-01T00:08:00.001015Z", "average": 93.75, "lasttime": "2017-05-01T00:08:00.001018Z", "maxvalue": 95, "firstvalue": 93, "mintime": "2017-05-01T00:08:00.001016Z", "lastvalue": 94, "countgood": 4, "countuncertain": 0, "countbad": 0, "sum": 375, "minvalue": 93, "maxtime": "2017-05-01T00:08:00.001017Z", "sd": 14.36 }, "temperature": { "firsttime": "2017-05-01T00:08:00.001015Z", "average": 44.25, "lasttime": "2017-05-01T00:08:00.001018Z", "maxvalue": 45, "firstvalue": 43, "mintime": "2017-05-01T00:08:.001012Z", "lastvalue": 44, "countgood": 4, "countuncertain": 0, "countbad": 0, "sum": 177, "minvalue": 43, "maxtime": "2017-05-01T00:08:00.001019Z", "sd": 14.36 }, "starttime": "2017-05-01T00:08:00.001Z", "endtime": "2017-05-01T00:08:00.002Z" }, { "pressure": { "firsttime": "2017-05-01T00:08:00.002015Z", "average": 95, "lasttime": "2017-05-01T00:08:00.2018Z", "maxvalue": 96, "firstvalue": 95, "mintime": "2017-05-01T00:08:00.002010Z", "lastvalue": 96, "countgood": 4, "countuncertain": 0, "countbad": 0, "sum": 380, "minvalue": 94, "maxtime": "2017-05-01T00:08:00.002019Z", "sd": 14.36 }, "temperature": { "firsttime": "2017-05-01T00:08:00.002014", "average": 44.5, "lasttime": "2017-05-01T00:08:00.2018Z", "maxvalue": 45, "firstvalue": 44, "mintime": "2017-05-01T00:08:00.002010Z", "lastvalue": 44, "countgood": 4, "countuncertain": 0, "countbad": 0, "sum": 178, "minvalue": 43, "maxtime": "2017-05-01T00:08:00.002011Z", "sd": 14.36 }, "starttime": "2017-05-01T00:08:00.002Z", "endtime": "2017-05-01T00:08:00.003Z" } ] ``` ### Get Aggregate Data for a Specific Time Range with Time Zone for Simulation Entity Specify the desired time range using the URL parameters from and to: ```http https://gateway.{region}-{environment}.{mindsphere-domain}/api/iottsaggregates/v3/aggregates/{assetId}/{aspectName}?from=2017-05-01T00:08:00.001%2B02:00&to=2017-05-01T00:08:00.003%2B02:00&intervalUnit=millisecond&intervalValue=1 ``` For setting the time zone of the time range, the `+` character must be encoded to `%2B` in the URL. For example, `2017-05-01T00:08:00.001%2B02:00` instead of `2017-05-01T00:08:00.001+02:00`. ## Defining Start Time and Time Range As described in [Requesting Aggregates](api-iottsaggregates-overview.html), the start and end times cannot be defined with higher precision than the time range. For clarification, the following table lists combinations of start time, end time, and interval length and explains the expected response: | Start Time | End Time | Interval Length | Returned Intervals | Explanation | | ---------------------------- | ---------------- | --------------- | ------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 14:00:01.000 | 14:00:01.300 | 1 ms | - | Rejected since the response would contain more than 200 intervals. *Interval length available for simulation entities only.* | | 14:00:01.005 | 14:00:01.015 | 10 ms | - | Rejected since start and end time defined with higher precision than interval length. *Interval length available for simulation entities only.* | | 14:00:01.000 | 14:00:02.000 | 1 second | 1 | *Interval length available for simulation entities only.* | | 14:00 | 15:00 | 1 minute | 60 | *Interval length available for performance entities only.* | | 14:00 | 15:00 | 4 minute | 15 | *Interval length available for performance entities only.* | | 14:00 | 15:00 | 2 minute | 30 | Calculated from pre-calculated intervals and time series data. *Interval length available for performance entities only.* | | 14:00 | 15:00 | 3 minute | 20 | Calculated from pre-calculated intervals and time series data. *Interval length available for performance entities only.* | | 14:01 | 15:01 | 2 minute | 30 | Calculated on the fly from raw time series data. *Interval length available for performance entities only.* | | 14:30 | 16:30 | 1 hour | - | Rejected - start and end times defined with higher precision than interval length | | 14:30 | 16:30 | 60 minute | 2 | *Interval length available for performance entities only.* | | 14:30 | 16:30 | 30 minute | 4 | *Interval length available for performance entities only.* | | March 1st, 00:00 | March 5th, 00:00 | 1 day | - | Rejected - start and end times defined with higher precision than interval length *Interval length available for performance entities only.* | | March 1st, 00:00 | March 5th, 00:00 | 24 h | 4 | *Interval length available for performance entities only.* | | March 1st (Wednesday), 00:00 | April 1st, 00:00 | 1 week | - | Rejected - start and end times defined with higher precision than interval length (assuming weeks start on Monday for this environment) *Interval length available for performance entities only.* | | March 1st, 00:00 | April 5th, 00:00 | 7 day | 5 | *Interval length available for performance entities only.* | | March 1st, 00:00 | June 1st, 00:00 | 1 month | 3 | *Interval length available for performance entities only.* | | March 1st, 00:00 | June 29th, 00:00 | 30 day | 4 | *Interval length available for performance entities only.* | # IoT Time Series Aggregates Service – V3.x to V4.x Migration Guide Announcement For more information about new features and limitations for Aggregate V4 API, refer [Version 4.0 - What's New?](api-iottsaggregates-version-v4-whats-new.html). The older version of Aggregate V3 will be available without any support and reduced quota till 30th June 2024. ## Forming Aggregate V4 Request Consider an example where there is an asset type `forklift` with an aspect `tireMonitor` and with the variables `pressure`, `temperature`, and `treadDepth`. The `pressure` variable has a quality code value of `Y` while the others are `N`. An instance of `forklift` with an asset ID of `978528e7a124458f87c8f1d38fd9400f` is defined. - Existing Aggregate V3.x URL: ```http https://gateway.{region}-{environment}.{mindsphere-domain}/api/iottsaggregates/v3/aggregates/978528e7a124458f87c8f1d38fd9400f/tireMonitor?from=2017-05-01T00:08:00Z&to=2017-05-01T00:16:00Z&select=pressure,temperature&intervalUnit=minute&intervalValue=4 ``` - New Aggregate V4.x URL: ```http https://gateway.{region}-{environment}.{mindsphere-domain}/api/iottsaggregates/v4/aggregates?assetId=978528e7a124458f87c8f1d38fd9400f&aspectName=tireMonitor&from=2017-05-01T00:08:00Z&to=2017-05-01T00:16:00Z&select=pressure,temperature&intervalUnit=minute&intervalValue=4 ``` There are differences in response which are mentioned below: The following response is returned for the Aggregate V3.x API: Response ```json [ { "pressure": { "firsttime": "2017-05-01T00:09:00Z", "average": 93.75, "lasttime": "2017-05-01T00:12:00Z", "maxvalue": 95, "firstvalue": 93, "mintime": "2017-05-01T00:10:00Z", "lastvalue": 94, "countgood": 4, "countuncertain": 0, "countbad": 0, "sum": 375, "minvalue": 93, "maxtime": "2017-05-01T00:11:00Z", "sd": 14.36 }, "temperature": { "firsttime": "2017-05-01T00:09:00Z", "average": 44.25, "lasttime": "2017-05-01T00:12:00Z", "maxvalue": 45, "firstvalue": 43, "mintime": "2017-05-01T00:09:00Z", "lastvalue": 44, "countgood": 4, "countuncertain": 0, "countbad": 0, "sum": 177, "minvalue": 43, "maxtime": "2017-05-01T00:11:00Z", "sd": 14.36 }, "starttime": "2017-05-01T00:08:00Z", "endtime": "2017-05-01T00:12:00Z" }, { "pressure": { "firsttime": "2017-05-01T00:13:00Z", "average": 95, "lasttime": "2017-05-01T00:16:00Z", "maxvalue": 96, "firstvalue": 95, "mintime": "2017-05-01T00:15:00Z", "lastvalue": 96, "countgood": 4, "countuncertain": 0, "countbad": 0, "sum": 380, "minvalue": 94, "maxtime": "2017-05-01T00:16:00Z", "sd": 14.36 }, "temperature": { "firsttime": "2017-05-01T00:13:00Z", "average": 44.5, "lasttime": "2017-05-01T00:16:00Z", "maxvalue": 45, "firstvalue": 44, "mintime": "2017-05-01T00:14:00Z", "lastvalue": 44, "countgood": 4, "countuncertain": 0, "countbad": 0, "sum": 178, "minvalue": 43, "maxtime": "2017-05-01T00:15:00Z", "sd": 14.36 }, "starttime": "2017-05-01T00:12:00Z", "endtime": "2017-05-01T00:16:00Z" } ] ``` The following response is returned for the Aggregate V4.x API: Response ```json { "aggregates":[ { "pressure": { "firsttime": "2017-05-01T00:09:00Z", "average": 93.75, "lasttime": "2017-05-01T00:12:00Z", "maxvalue": 95, "firstvalue": 93, "mintime": "2017-05-01T00:10:00Z", "lastvalue": 94, "countgood": 4, "countuncertain": 0, "countbad": 0, "sum": 375, "minvalue": 93, "maxtime": "2017-05-01T00:11:00Z", "sd": 14.36 }, "temperature": { "firsttime": "2017-05-01T00:09:00Z", "average": 44.25, "lasttime": "2017-05-01T00:12:00Z", "maxvalue": 45, "firstvalue": 43, "mintime": "2017-05-01T00:09:00Z", "lastvalue": 44, "countgood": 4, "countuncertain": 0, "countbad": 0, "sum": 177, "minvalue": 43, "maxtime": "2017-05-01T00:11:00Z", "sd": 14.36 }, "starttime": "2017-05-01T00:08:00Z", "endtime": "2017-05-01T00:12:00Z" }, { "pressure": { "firsttime": "2017-05-01T00:13:00Z", "average": 95, "lasttime": "2017-05-01T00:16:00Z", "maxvalue": 96, "firstvalue": 95, "mintime": "2017-05-01T00:15:00Z", "lastvalue": 96, "countgood": 4, "countuncertain": 0, "countbad": 0, "sum": 380, "minvalue": 94, "maxtime": "2017-05-01T00:16:00Z", "sd": 14.36 }, "temperature": { "firsttime": "2017-05-01T00:13:00Z", "average": 44.5, "lasttime": "2017-05-01T00:16:00Z", "maxvalue": 45, "firstvalue": 44, "mintime": "2017-05-01T00:14:00Z", "lastvalue": 44, "countgood": 4, "countuncertain": 0, "countbad": 0, "sum": 178, "minvalue": 43, "maxtime": "2017-05-01T00:15:00Z", "sd": 14.36 }, "starttime": "2017-05-01T00:12:00Z", "endtime": "2017-05-01T00:16:00Z" } ] } ``` ## Related links - [Aggregates V4.x Overview](api-iottsaggregates-overview-v4.html) - [Aggregates V4.x Samples](api-iottsaggregates-samples-v4.html) - [Aggregates V4.x Basic Parameters](api-iottsaggregates-basics-parameters-v4.html) ## FAQs 1. Why should I migrate to V4?\ Aggregate v4 API centers around improved usability through advanced intelligent features and new functionalities such as: - All query parameters are optional except `assetId` and `aspectName` - If not provided by user, parameter `intervalValue` and `intervalUnit` gets calculated automatically using `from` and `to` date - Added new parameter `count` to specify the number of aggregates a user needs - Asset and Aspect ids as query parameter - Support for ISO date formats - Removed limit on records and added limits on duration and number of calls 1. Is there any difference between v3 and v4 API?\ Yes there are below mentioned differences: - Any value of interval is not supported but aggregates are now available where user just needs to specify time range based on which optimal intervals are auto selected by the system to provide response swiftly. - There is no limitations of 200 records in new V4 API as compared to V3 API - Enhanced ISO date format is now supported in Aggregate V4 APIs - Simulation asset is not supported on Aggregate V4 API The IoT Time Series (TS) Aggregates Service is designed to quickly and easily facilitate the visualization of time series data on Industrial IoT. This service or API creates aggregated summaries of numeric time series data and provides interfaces to read them. As a result, this allows applications to retrieve smaller data sets that cover a longer time period with better performance than processing all the raw time series data. For example, an aspect could create new data every second, which adds up to ~2.5 million records per month. An application could use the IoT TS Aggregates Service to get a summary for each day of the month, obtaining only 30 records. The focus of version 4.x centers around improved usability through advanced intelligent features and new functionalities such as: - All query parameters are optional except `assetId` and `aspectName` - If not provided by user, parameter `intervalValue` and `intervalUnit` gets calculated automatically using `from` and `to` date - Added new parameter `count` to specify the number of aggregates a user needs - Asset and Aspect ids as query parameter - Support for ISO date formats - Removed limit on records and added limits on duration and number of calls Limits are introduced on the time frame of data returned and the number of calls per minute allowed in the system to keep consistent performance of the response and continuity of the service. | Limits | V3 api | V4 api | | ------------------------------------------- | ------ | ------ | | Maximum 200 aggregate results in response | Yes | No | | Internal limit of 5000 records | Yes | No | | 'minute' aggregate limit : 48 hours | No | Yes | | 'hour' aggregate limit : 31 days | No | Yes | | 'day/week/month' aggregate limit : 366 days | No | Yes | | Functionality | V3 | V4 | Comment | | -------------------- | --- | --- | ------------------- | | On the fly | Yes | No | | | Simulation Asset | Yes | No | | | Date format Handling | No | Yes | Enhanced ISO format | It is recommended to migrate the applications and use the newer features of the version 4.x. For information on how to use the 4.x API, refer to [Samples for v4.x](api-iottsaggregates-samples-v4.html). # IoT Times Series Bulk Service – API Overview ## Region Europe 1 [Download OpenAPI Specification](/insights-hub/api_specs/iottsbulk-v3-9-0-xcelerator.yaml) # IoT Time Series Bulk Service ## Idea The IoT Time Series Bulk Service is a variant to the IoT Time Series Service. It is designed for importing big amounts of data into Insights Hub in bulk. ## Access For accessing this service, you need to have the respective roles listed in [IoT Time Series Service roles and scopes](https://documentation.mindsphere.io/MindSphere/concepts/concept-roles-scopes.html#iot-time-series). For accessing the Secure Data Sharing (SDS) protected APIs, you need to have appropriate Policy Definitions in place. For the list of supported APIs and required actions, refer to the [Required Policy actions by Industrial IoT services](../core-resourceaccessmanagement/api-resourceaccessmanagement-services-actions-map.html#iot-time-series-service) section. ## Basics In contrast to the [IoT Time Series Service](../iot-iottimeseries/api-iottimeseries-overview.html), this service ingests data in batch mode and can return more than 2,000 items per response when reading the stored data. It returns as many items as possible until size and time limitations on the [Industrial IoT Gateway](https://documentation.mindsphere.io/MindSphere/concepts/concept-gateway-url-schemas.html#restrictions) are exceeded. If it cannot return all requested data in a single response, a link for requesting the next data set is added to the response body. The URL of this link is based on the already returned data and the original request. The start time is set to the timestamp of the last returned record and the maximum number of records is reduced by the number of already returned records. The IoT Time Series Bulk Service accepts up to 100 consecutive files per import job. For performance asset, all the timestamps should belong to the same day (for example, FROM `2021-02-18T00:00:00.001Z` TO `2021-02-19T00:00:00.000Z`). For Bulk Ingest of time series, only time stamps in ISO format are supported. Important The following practices are suggested for best results for Performance Assets while using IoT Timeseries Bulk Ingest Service. - Combine complete data for Performance Asset in single/multiple file for any Asset, Aspect and upload it in single job for bulk processing. - Include as much data in single job. - Bulk Ingest API is designed to upload historical data only unless data size of an individual job is in few MB's. - Small job sizes (Less than ~ 20KB) should be avoided to avail optimal performance. - Bulk Ingest jobs will be processed asynchronously. Info The import requests are processed in batch mode running periodically. Attention Previously stored time series data is overwritten when uploading new data for the same time period. ## Features The IoT Time Series Bulk Service exposes its API for realizing the following tasks: - Import time series data for performance assets in bulk - Based on Timeseries data density, the maximum number of record values for the specified time range are obtained within the 1 minute overall request time limit. It is recommended to use this while reading bulk data which can withhold response time up to 1 minute. ## Usage Quota and Limits The usage quota and limits details for the IoT Time Series services is mentioned in [API rate limits](../iot-iottimeseries/api-iottimeseries-apiratelimits.html). ## Limitations - Currently, only the IoT File Service is supported as data source. - Only JSON file format is supported as input file. - The maximum timerange for READ operation for getting bulk timeseries data is 90 days. - The maximum volume of time series data cannot exceed 1 GB for performance asset on Public Cloud. The maximum volume will be 100 MB for Private Cloud. - Import jobs for performance assets are allowed for data in future upto 7 days only. - For Private cloud, Import jobs for performance assets are allowed for data older than 30 minutes only. - For Timeseries Data ingestion of past 7 days, use either IoT TS Bulk Services or PUT endpoint of [Timeseries services](https://developer.mindsphere.io/apis/iot-iottimeseries/api-iottimeseries-api.html). If both the methods are used, data consistency will be compromised. - Timeseries data will not be retrieved via GET endpoint of [Timeseries services](https://developer.mindsphere.io/apis/iot-iottimeseries/api-iottimeseries-api.html) for Data Ingested via IoT TS Bulk Services for past 7 days. - The data in json file in Import jobs needs to be in ascending timestamp order. - Import jobs for performance assets are rejected if timestamps with higher precision than milliseconds are used. - The `+` character used to include timezone in timestamp must be encoded to `%2B` in time range. For example, `2017-09-21T14:08:00%2B02:00` instead of `2017-09-21T14:08:00+02:00`. - Variable with Data type as `BIG_STRING` is not supported by IoT TS Bulk Services. - Bulk Service Data ingestion will be processed within 4 hours in most of the cases. This time may increase based on load. - The Bulk Service for Read Operations for `Get/Timeseries/{entity}/{propertySetName}` will have a response time up to 1 minute based on query range and data density. - The Bulk Service for Read Operations for `Get/Timeseries/{entity}/{propertySetName}` will not be retrieved upto 4 hours in most of the cases, if Timeseries data is ingested via PUT Endpoint of [Timeseries Services](https://developer.mindsphere.io/apis/iot-iottimeseries/api-iottimeseries-api.html). - IoT Timeseries bulk Import is not supported for Azure Virtual Private Cloud environment ## Example Scenario A company is already collecting data for an existing field device. They want to correlate this with simulation data created during the engineering or testing process in order to derive relevant KPIs. The company uploads the simulation data using the IoT File Service and triggers the import with the IoT Time Series Bulk Service. After the importing the data can be consumed via this service to be used by an application that also has access to the Time Series data of the field device. # IoT Time Series Bulk Service – Samples ## Importing Bulk Data To import bulk data from the IoT File Service, use the following endpoint: ```http POST /api/iottsbulk/v3/importJobs ``` Sample request: ```http POST /api/iottsbulk/v3/importJobs HTTP/1.1 Host: gateway.eu1.mindsphere.io Content-Type: application/json Accept: application/json Authorization: Bearer {token} { "data": [ { "entity": "7dede1074eb759611c306606a66d0ca6", "propertySetName": "front_left_tire", "timeseriesFiles": [ { "filePath": "my_path/my_file_1.json", "from": "2018-09-21T15:00:01.787000Z", "to": "2018-09-21T15:05:50.633015Z" }, { "filePath": "my_path/my_file_2.json", "from": "2018-09-21T15:07:00.795015Z", "to": "2018-09-21T15:11:40.417015Z" } ] } ] } ``` After validation checks, the service returns a location header with the associated import job ID. ## Checking the Import Status The import job ID can be used for import status checks, e.g.: ```http GET /api/iottsbulk/v3/importJobs/job123 HTTP/1.1 Host: gateway.eu1.mindsphere.io Accept: application/json Authorization: Bearer {token} ``` Sample response: ```json [ { "id": "job123", "status": "SUBMITTED", "message": "{message}", "startTime": "2018-11-01T14:48:37.056Z", "lastModified": "2018-11-01T14:48:37.056Z" } ] ``` ## Retrieving Bulk Data To retrieve bulk data from the IoT File Service, use the following endpoint: ```http https://gateway.{region}.{mindsphere-domain}/api/iottsbulk/v3/timeseries/{asset_id}/{aspect_name} ``` Sample request: ```http GET /api/iottsbulk/v3/timeseries/my_vehicle_123/front_left_tire?from=2018-11-01T10:00:00.050000Z&to=2018-11-01T10:00:05.000000Z&select=pressure,temperature HTTP/1.1 Host: gateway.eu1.mindsphere.io Accept: application/json Authorization: Bearer {token} ``` Sample response: ```json { "records": [ { "pressure": 94, "pressure_qc": 192, "temperature": 45, "_time": "2018-11-01T10:00:00.051035Z" }, { "pressure": 95, "pressure_qc": 192, "temperature": 45, "_time": "2018-11-01T10:00:00.052070Z" }, ... ], "nextRecord": "https://gateway.{region}.mindpshere.io/api/iottsbulk/v3/iottimeseries/my_vehicle_123/front_left_tire?from=2018-11-01T10:00:02.000000Z&to=2018-11-01T10:00:05.000000&select=pressure,temperature" } ``` ## Deleting Bulk Data Bulk data is deleted using the IoT Time Series Service via the following endpoint: ```http DELETE /api/iottimeseries/v3/timeseries/{asset_id}{aspect_name} ``` Sample request: ```http DELETE /api/iottimeseries/v3/timeseries/my_vehicle_123/front_left_tire?from=2018-11-01T10:00:00Z&to=2018-11-01T11:00:00Z HTTP/1.1 Host: gateway.eu1.mindsphere.io Accept: application/json Authorization: Bearer {token} ``` Info Currently, only full hours of bulk data can be deleted and the time range must start and end at the top of an hour. # IoT Times Series Stream Registration Service – API Overview ## Region Private Cloud [Download OpenAPI Specification](/insights-hub/api_specs/iottsstreamregistration-v3-0-3-xcelerator.yaml) # IoT Timeseries Data Streaming ## Overview Currently, to fetch Time Series data, an Insights Hub application would have to query all Asset instances, compare the timestamps, find the latest Time Series data and update the application. This would involve multiple requests through Industrial IoT Gateway which is expensive and not optimal or customer friendly. With Timeseries data streaming, the 3rd party applications will get actual timeseries data streamed by Insights Hub on new Time Series data arrival for concrete Assets in near real time. This mechanism enables customer to directly consume Time series data in their application. This improves the usability of Insights Hub Services and user experience for the developers. Note IoT Timeseries Data Streaming will be available only in Early Access (EA) for Local Private Cloud customer. The feature can be provisioned via offering of 21000 for respective customer tenant. Contact the customer support for availing this feature. ## Streaming output The streaming component will receive the data in following format: - owner `(String)`: It is represented as - tenantId/assetId/aspectName - registrationId `(String)`: It is the unique ID for the validated registration. The registrationId can be used to segregate the data. - data: Data is in `List>` format and the order will be as ingested into timeseries. ```json { "owner": "siemens/28068dafbf7046d2a7a2be4bccb49409/status", "registrationId": "c19677f8-e215-4a56-975d-8ee7f3ba4538", "data": [ { "_time": "2024-03-27T01:21:00Z", "Pressure": 22 }, { "_time": "2024-03-27T01:22:00Z", "Pressure": 21, "Status": true, "Remarks": "all_good" }, { "_time": "2024-03-27T01:23:00Z", "Pressure": 24, "Temperature": 102 } ] } ``` ## Limitations - Only asset level registrations are supported. - Only 10 registrations are allowed per tenant with maximum 50 assets under single registration. - Customer can have up to 5 destinations per tenant. - Only Kafka is supported as the destination streaming component. - Updating registration is not supported. Customer needs to delete and recreate registration. In case of key/secret rotation, customer needs to delete and recreate the registration, and validate the same. - Same assetId cannot be used in or across multiple registrations. - There can be multiple issues where the validation of a stream registration can fail (BROKEN), the same will be specified in the error field. If it has happened due to timeouts, a simple retry would suffice, otherwise user needs to delete and re-create the registration. - Asset deletion will not be reflected in registrations. Deleted assets may still be visible in registration but the data will not be streamed. - Any changes in aspect/variable metadata (rename/update/delete) may not be reflected in output data for 30 minutes. ## Setting up the destination Customer can create a kafka stream in the same kafka cluster in customer account or can deploy a kafka cluster on an external server. - In case of kafka setup in same cluster as the customer account, the customer needs to only provide the zookeeper broker address and stream name while creating the registration. - In case of a kafka cluster created in external server, - Customer needs to provide a publicly accessible route. - Customer needs to take care of the encryption at transit and other security of the kafka cluster. - It is recommended to follow the best practices for optimum performance. ## FAQs 1. What will be the delay in getting timeseries data after ingestion? - The delay will depend on the intermediate processing and validation at various stages at Edge, Connectivity and IoT timeseries streaming before it is forwarded to customer's streaming component. It may vary case to case, however the functionality is built in such a way it will be near real time. 1. Is there is retry mechanism if the customer destination component is somehow not able to receive data? - The functionality internally performs an exponential backoff retry for failed messages. If the delivery fails despite that, the registration is marked BROKEN. 1. What if customer is interested for specific variable's data to be streamed? - Customer needs to filter out variable's data at recipient end to consume it in respective use-case. # Resources This section lists all currently published resources for Insights Hub developers. It is updated regularly while the Insights Hub ecosystem is continuously being enriched. | Name | Description | | --------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | [Insights Hub Monitor Plugin SDK](insights-hub-monitor-plugin-sdk/index.html) | The Insights Hub Monitor Plugin SDK enables web developers to create plugins for Insights Hub Monitor. | | [MindConnect Library v3](mindconnect-lib/resources-mclib-overview.html) | The MindConnect Library v3 is a collection of functionalities which allows a client application to connect and exchange data with Insights Hub. | | [MindConnect Library v4](mindconnect-lib-v4/resources-mclib-overview.html) | The MindConnect Library v4 is the successor of MindConnect Library v3 with a new architecture improving performance, reducing footprint and simplifying user customization. You can download the MindConnect Library from the [Siemens Industry Online Support (SIOS)](https://support.industry.siemens.com/cs/ww/en/view/109772086) or [clone it from GitHub](https://github.com/mindsphere/mindconnect-lib). | | [OS Bar](https://documentation.mindsphere.io/MindSphere/resources/osbar/getting-started.html) | The OS Bar connects all Insights Hub applications on a UI level. It can be used for showing the application name and routing back to the launchpad. | | [Industrial IoT SDK for Java](industrial-iot-sdk-java/index.html) | The Industrial IoT SDK for Java enables Java developers to work with Insights Hub easily. It provides a Java client implementation for the RESTful Industrial IoT APIs. | | [Industrial IoT SDK V2 for Java](industrial-iot-sdk-java-v2/index.html) | The Industrial IoT SDK Version 2 for Java enables Java developers to work with the Insights Hub easily using service specific SDKs. The SDK is generated automatically using the Industrial IoT Service's specification yaml files to provide Java client implementation for the RESTful Industrial IoT APIs. | | [Industrial IoT SDK for Node.js](industrial-iot-sdk-node/index.html) | The Industrial IoT SDK for Node.js enables Node.js developers to work with the Insights Hub easily using service specific SDKs. The SDKs are generated using the Industrial IoT Service API specifications to provide Node.js client implementations for the RESTful Industrial IoT Service APIs. | | [Industrial IoT SDK for Python](industrial-iot-sdk-python/index.html) | The Industrial IoT SDK for Python enables Python developers to work with Insights Hub easily using service specific SDKs. The SDK is generated automatically using the Industrial IoT Service's specification yaml files to provide Python client implementation for the RESTful Industrial IoT APIs. | | [Web Components for Insights Hub and Industrial IoT](https://documentation.mindsphere.io/MindSphere/resources/webcomponents/index.html) | The Web Components for Insights Hub and Industrial IoT enables a Web developer to use out of the box components which can be integrated easily in a web application. There exist easy frontend component like DateTimeRange picker as also components which involve the Insights Hub backend services like the Asset View component. | | [Open Edge Device Kit](https://documentation.mindsphere.io/MindSphere/resources/openedge-devicekit/index.html) | Open Edge Device Kit is a stand-alone module for third-party device producers that can be installed on the device and acts as an interface to Insights Hub. | ## Insights Hub Open Source Tools and Libraries Insights Hub Open Source Tools and Libraries are contributions from the community. These are demo applications, libraries and SDKs and useful connectivity tools which are maintained and released as open source software. The source code for all libraries is available on Github and the packages can be downloaded via common package management tools. The full documentation can be found at [Siemens Developer Portal](https://developer.siemens.com/industrial-iot-open-source/overview.html). | Name | Description | | ---------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | [MindConnect-NodeJS](https://developer.siemens.com/industrial-iot-open-source/mindconnect-nodejs/index.html) | This is a nodejs library which can be used as a starting point for custom agent implementation in node.js. The library comes with support for TimeSeries, Events and File Upload and it can create and update the DataSourceConfiguration and the Mappings in Insights Hub. The most recent feature is support for automatic mappings to the asset instances. | | [Industrial IoT TypeScript SDK](https://developer.siemens.com/industrial-iot-open-source/mindconnect-nodejs/sdk/index.html) | TypeScript/JavaScript Community SDK for Industrial IoT APIs. It implements support for both browser (e.g. angular, react...) and backend development in node.js and it supports all different Insights Hub authentication types (Cookies, UserCredentials, AppCredentials, ServiceCredentials, Insights Hub Agents). It is packaged as part of @mindconnect/mindconnect-nodejs library. | | [Insights Hub Command Line Interface](https://developer.siemens.com/industrial-iot-open-source/mindconnect-nodejs/cli/index.html) | Insights Hub GUIs are great but some tasks are still better done from the command line. The Insights Hub CLI is a power tool for experienced Insights Hub administrators as it gives you scriptable access to common tasks like agent onboarding and offboarding, asset management, historical time series upload, user management and it also offers valuable tooling for software developers. It is packaged as part of @mindconnect/mindconnect-nodejs library. | | [MindConnect Node-RED Node](https://developer.siemens.com/industrial-iot-open-source/node-red-contrib-mindconnect/index.html) | A Node-RED node which can be used to upload the time series data, files and events to Insights Hub. The node runs on x86, Raspberry PI, SIMATIC IOT 2040 and is also available as a docker container on dockerhub. There are also live flow examples with flows sending e.g. the OPC UA or MQTT Data to Insights Hub at | | [Insights Hub Development Proxy](https://developer.siemens.com/industrial-iot-open-source/mindconnect-nodejs/cli/development-proxy.html) | The development proxy can be used to kickstart your Insights Hub development. It provides an endpoint at your local machine which will authenticate all requests to the Industrial IoT APIs. The proxy is packaged as part of @mindconnect/mindconnect-nodejs library. | | [Insights Hub Authentication Helper](https://developer.siemens.com/industrial-iot-open-source/mindsphere-auth-helper/index.html) | This chrome extension can be used to simplify the development tasks for which you need to use the SESSION and XSRF-TOKEN to access Industrial IoT APIs. It provides an easy way to copy the Insights Hub authentication cookies to the clipboard without having to go to chrome developer tools. | | [JSON Schema to Insights Hub Demo App](https://github.com/mindsphere/json-schema-to-mindsphere-demo-app) | Demo Application showing how to create Insights Hub modelling artifacts (Variables, Aspect- and Asset-Types as well as assets) straight from a JSON schema. See more on [GitHub](https://github.com/mindsphere/json-schema-to-mindsphere-demo-app) and on [Medium](https://medium.com/@sn0wcat_92713/how-to-create-mindsphere-artefacts-from-json-schema-definitions-1b2f3e446f6a). | | [MindConnect Library V4 Demo Agent](https://github.com/mindsphere/mindconnect-MCL-C-agent) | Agent for Insights Hub based on Mindconnect Library V4 and written in C. Compiled and tested on Raspberry Pi V4 with Raspbian Buster.See more on [GitHub](https://github.com/mindsphere/mindconnect-MCL-C-agent). | | [Visual Flow Creator Examples](https://github.com/mindsphere/vfc-examples) | Example flows for [Visual Flow Creator](https://documentation.mindsphere.io/resources/html/visualflow-creator/en-US/index.html). The examples include the cloud to device communication and commanding via MQTT, creation of the custom API endpoints in the Insights Hub and more. The full list of examples is available on [GitHub](https://github.com/mindsphere/vfc-examples). | | [Siemens Industrial Edge](https://github.com/industrial-edge) | Open source projects and samples for the open [Industrial Edge](http://siemens.com/industrial-edge) computing platform from Siemens. The full list of examples is available on [GitHub](https://github.com/industrial-edge). | | [Open Source Documentation](https://developer.siemens.com/industrial-iot-open-source/overview.html) | The open source documentation is available as an open source [GitHub](https://github.com/mindsphere/mindsphere.github.io) project | # Industrial IoT SDK for Java ## Introduction The Industrial IoT SDK for Java enables Java developers to work with Insights Hub easily. It provides a Java client implementation for the RESTful Industrial IoT APIs. The Industrial IoT SDK for Java provides client implementations for the following APIs: | Name | API Version | | ---------------------------------------------------------- | ----------- | | [Asset Management](apidocs/AssetManagement.html) | 3.4 | | [IoT Time Series](apidocs/IOT_Timeseries.html) | 3.1 | | [IoT TS Aggregates](apidocs/IOT_Timeseries_Aggregate.html) | 3.0 | | [IoT File](apidocs/IOT_File.html) | 3.1 | | [Event Management](apidocs/EventManagement.html) | 3.5 | | [Event Analytics](apidocs/EventAnalytics.html) | 3.0 | See the [Getting Started](sdkreadme.html) for installation and configuration of an API client. ## Get the SDK You can download the Industrial IoT SDK for Java from the *Siemens Industry Online Support (SIOS) Portal* by following these links [**[EN]**](https://support.industry.siemens.com/cs/ww/en/view/109757603)/[**[DE]**](https://support.industry.siemens.com/cs/ww/de/view/109757603). ## Features ### Client Configuration You can easily configure the Industrial IoT SDK for Java through configuration parameters such as proxy settings and connection time-outs. ### Authorization Handling Mechanism The Industrial IoT SDK for Java provides an easy authorization handling mechanism. Developers can configure user authorization tokens or service credentials. The SDK provides multiple ways for setting up service credentials for use in an API client. Environment variables can be set for service credential configuration to generate authorization for API calls. ### Page Iterators This SDK for Java provides page iterator objects for easy pagination for most API clients. Developer applications can configure paging parameters and get next/previous or specific page data. ### Logging The Industrial IoT SDK for Java logging is configured with SLF4J, which is an abstraction layer that enables the use of any logging framework at runtime. Supported logging systems include the Java Logging Framework and Apache Log4j, among others. The Industrial IoT SDK logging can be enabled or disabled by the developer application. See [Logging](sdk_logging.html) for more information. ### Exception Handling This SDK provides an exception handling mechanism. The SDK exceptions are structured to identify client and server exceptions. Information such as error code, message, HTTP status, logref and cause can help to easy identify exceptions. See [Exception Handling](exception_handling.html) for more information. # Industrial IoT SDK for Java - Exception Handling Understanding how and when the Industrial IoT SDK for Java throws exceptions is important to build high-quality applications using the SDK. The following sections describe the different scenarios in which exceptions are thrown by the SDK and how to handle them appropriately. ## MindsphereException [`MindsphereException`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/core/exception/MindsphereException.html) is the most common exception that you will experience when using the Industrial IoT SDK for Java. This exception represents an error response from an Industrial IoT service. MindsphereException provides you with information such as: - Returned HTTP status - Returned HTTP status Code - Detailed error message from the service - Reference to the error occurred ### Structure of MindsphereException | Name | Description | Type | | --------------- | ------------------------------------- | ---------- | | errorStatusCode | Error code (if returned by API) | String | | errorMessage | Error message | String | | httpStatus | HTTP status | HttpStatus | | logref | Logger reference (if returned by API) | String | | cause | Cause of the exception (if any) | Throwable | ### Subclasses of MindsphereException - [MindsphereServiceException](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/core/exception/MindsphereServiceException.html) - [MindsphereClientException](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/core/exception/MindsphereClientException.html) - [MindsphereClientConfigurationException](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/core/exception/MindsphereClientConfigurationException.html) - [MindsphereForbiddenAccessException](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/core/exception/MindsphereForbiddenAccessException.html) [MindsphereClientException](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/core/exception/MindsphereClientException.html) is a subclass of [MindsphereException](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/core/exception/MindsphereException.html). It is thrown when an error occurs due to user's fault (a request with missing values). For example, if user tries to call an API with null values for mandatory parameters then MindsphereClientException is thrown to indicate that required parameters are missing. [MindsphereServiceException](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/core/exception/MindsphereServiceException.html) is a subclass of [MindsphereException](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/core/exception/MindsphereException.html). It is thrown when an error occurs at the API endpoint. For example, if user tries to call an API with invalid values for mandatory parameters. A MindsphereServiceException is thrown only when an error is encountered at the API endpoint. [MindsphereClientConfigurationException](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/core/exception/MindsphereClientConfigurationException.html) is a subclass of [MindsphereClientException](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/core/exception/MindsphereClientException.html). It is thrown when incorrect configuration parameters are set. For example, if a developer uses an incorrect value for the proxy, MindsphereClientConfigurationException is thrown to indicate that incorrect proxy host or proxy port. [MindsphereForbiddenAccessException](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/core/exception/MindsphereForbiddenAccessException.html) is a subclass of [MindsphereClientException](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/core/exception/MindsphereClientException.html). It is thrown when valid token is not retrieved with the provided credentials. # Industrial IoT SDK for Java – Logging ## Introduction Logging of the Industrial IoT SDK for Java is implemented with SLF4J (Simple Logging Facade for Java), which gives access to many logging frameworks such as Log4j or java.util.logging. This allows the end user to plug-in the desired logging framework. This section shows how to use Log4j or Logback with the SDK's logging functionality without making any change in application code. Attention Do not enable SDK logging for Industrial IoT SDK for Java version 1.1.0 as it logs service credentials. If it is already enabled, disable it. To disable, refer to section [Disable logging](#disable-logging). ## SDK Logging with Log4j or Logback ### Preparation #### Prepare Log4j for Industrial IoT SDK Logging If you want to use Log4j for Industrial IoT SDK logging, you need to download [SLF4J-Log4J12](https://mvnrepository.com/artifact/org.slf4j/slf4j-log4j12) and [Log4J](https://mvnrepository.com/artifact/log4j/log4j) jars. Build the project and ensure the jars are available as project dependencies. The configuration file `log4j.properties` must be available at the root resource folder `src\main\resources`. #### Prepare Logback for Industrial IoT SDK Logging If you want to use Logback for Industrial IoT SDK logging, you need to add the [logback-classic](https://mvnrepository.com/artifact/ch.qos.logback/logback-classic) jar to the project dependency. The configuration file `logback.xml` must be available at the root resource folder `src\main\resources`. Note The `logback-classic` jar is implicitly available, if the application is a spring-boot-starter project. ### Enable Logging and Set Log Levels Log levels have to be configured in the configuration file of your logging framework. The Industrial IoT SDK supports the log levels `DEBUG`, `INFO` and `ERROR`. The following examples illustrate how to set the rootLogger to `DEBUG`, which causes info, warning and error messages from all loggers in the application to be logged: ```bash # set log level in the log4j.properties file log4j.rootLogger=DEBUG, stdout log4j.appender.stdout=org.apache.log4j.ConsoleAppender log4j.appender.stdout.layout=org.apache.log4j.PatternLayout log4j.appender.stdout.layout.ConversionPattern=%d %p (%t) [%c] - %m%n ``` ```xml %d{yyyy-MM-dd HH:mm:ss} [%thread] %-5level %logger{36} - %msg%n ``` The following examples show how to set the log level only for the Industrial IoT SDK: ```bash #set log level in the log4j.properties file log4j.logger.com.siemens.mindsphere.sdk = INFO ``` ```xml ``` ### Disable logging Industrial IoT SDK logs can be disabled by setting the logging level to `OFF` in the configuration file: ```bash #disable logging in the log4j.properties file log4j.logger.com.siemens.mindsphere.sdk = OFF ``` ```xml ``` Spring-boot-starter projects If your application is a spring-boot-starter project, exclude the `logback-classic` and `log4j-over-slf4j` modules in the dependencies section of the project's `build.gradle` as shown below. This is to ensure that the `log4j.properties` file is considered as the logging configuration file. ```javascript // Use only if `log4j-over-slf4j` and `logback-classic` are available in project dependencies compile ('org.springframework.boot:spring-boot-starter-web') { exclude module : 'log4j-over-slf4j' exclude module: 'logback-classic' } ``` # Industrial IoT SDK for Java – Getting started ## Prerequisites - Java 8 - Gradle or Maven as build tool - User authorization token or service credentials with required scopes for APIs - Environment variable `HOST_ENVIRONMENT` set to current region. ```text env: HOST_ENVIRONMENT: {region} ``` If not specified, `HOST_ENVIRONMENT` will default to `eu1`. - In case you are using service credentials, they can be set as environment variables, so the client can fetch a token itself. - `MINDSPHERE_CLIENT_ID`\ Specify service credential ID - `MINDSPHERE_CLIENT_SECRET`\ Specify service credential - `MINDSPHERE_TENANT`\ Specify tenant name Note Setting environment variables for service credentials is not mandatory. Service credentials or user tokens can be set as parameters in the [MindsphereCredentials](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/auth/model/MindsphereCredentials.html) object as explained below in [API client and credentials configuration](#api-client-and-credentials-configuration). Attention The developer/customer is responsible for keeping the credentials safe. It is the decision of the developer/customer, whether it is safe enough to supply the credentials via environment variables. Requesting service credentials For requesting service credentials, see [Create Service Credentials](https://documentation.mindsphere.io/MindSphere/howto/howto-selfhosted-api-access.html#creating-service-credentials). ## Installation Instructions ### Downloading the Industrial IoT SDK [Download](index.html#get-the-sdk) the Industrial IoT SDK for Java from the *Siemens Industry Online Support (SIOS) Portal*. The `jar` and `pom` files of the downloaded archive have the following structure: ```text mindsphere\mindsphere-sdk\x.y.z\mindsphere-sdk-x.y.z.jar mindsphere\mindsphere-sdk\x.y.z\mindsphere-sdk-x.y.z-javadoc.jar mindsphere\mindsphere-sdk\x.y.z\mindsphere-sdk-x.y.z.pom ``` Note The group id for the Industrial IoT SDK for Java artifact is changed from `mindsphere` to `com.siemens.mindsphere` from version 1.2.0. Update the archive structure as follows: ```text com\siemens\mindsphere\mindsphere-sdk\x.y.z\mindsphere-sdk-x.y.z.jar com\siemens\mindsphere\mindsphere-sdk\x.y.z\mindsphere-sdk-x.y.z-javadoc.jar com\siemens\mindsphere\mindsphere-sdk\x.y.z\mindsphere-sdk-x.y.z.pom ``` Note `x.y.z` is the version number of the Industrial IoT SDK for Java (e.g. `1.0.0`). The file `mindsphere-sdk-x.y.z.pom` is required for downloading the transitive dependencies of the Industrial IoT SDK for Java. ### Adding SDK Dependency for Projects using Maven as a Build Tool - Create SDK folder structure in your local Maven repository: ```text $PATH\mindsphere\mindsphere-sdk\x.y.z\mindsphere-sdk-x.y.z.jar $PATH\mindsphere\mindsphere-sdk\x.y.z\mindsphere-sdk-x.y.z-javadoc.jar $PATH\mindsphere\mindsphere-sdk\x.y.z\mindsphere-sdk-x.y.z.pom $PATH\com\siemens\mindsphere\mindsphere-sdk\x.y.z\mindsphere-sdk-x.y.z.jar $PATH\com\siemens\mindsphere\mindsphere-sdk\x.y.z\mindsphere-sdk-x.y.z-javadoc.jar $PATH\com\siemens\mindsphere\mindsphere-sdk\x.y.z\mindsphere-sdk-x.y.z.pom ``` where `$PATH` is: - On Mac : `~/.m2/repository` - On Windows : `C:\Users\{username}\.m2\repository` - On Linux : `/home/{User_Name}/.m2/repository` - Add dependency in `pom.xml`: ```xml mindsphere mindsphere-sdk x.y.z com.siemens.mindsphere mindsphere-sdk x.y.z ``` ### Adding SDK Dependency for Projects using Gradle as a Build Tool - Create SDK folder structure in your local Maven repository or create a similar folder structure anywhere in your system: ```text $PATH\mindsphere\mindsphere-sdk\x.y.z\mindsphere-sdk-x.y.z.jar $PATH\mindsphere\mindsphere-sdk\x.y.z\mindsphere-sdk-x.y.z-javadoc.jar $PATH\mindsphere\mindsphere-sdk\x.y.z\mindsphere-sdk-x.y.z.pom $PATH\com\siemens\mindsphere\mindsphere-sdk\x.y.z\mindsphere-sdk-x.y.z.jar $PATH\com\siemens\mindsphere\mindsphere-sdk\x.y.z\mindsphere-sdk-x.y.z-javadoc.jar $PATH\com\siemens\mindsphere\mindsphere-sdk\x.y.z\mindsphere-sdk-x.y.z.pom ``` where `$PATH` is: - On Mac : `~/.m2/repository` - On Windows : `C:\Users\{username}\.m2\repository` - On Linux : `/home/{User_Name}/.m2/repository` - Advice gradle to look at the local Maven repository in `build.gradle` of your project as follows: ```text repositories { mavenLocal() mavenCentral() // Use this if SDK jars are not placed in local maven repository. maven { url file('{Absolute path of the created repository folder}') } } ``` - Add dependency in `build.gradle`: ```text compile 'mindsphere:mindsphere-sdk:x.y.z' compile 'com.siemens.mindsphere:mindsphere-sdk:x.y.z' ``` Once the build is complete, all transitive dependencies of the Industrial IoT SDK for Java are downloaded to your application. ## API Client and Credentials Configuration The lowest-level building blocks of the API are [`RestClientConfig`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/core/RestClientConfig.html) and [`MindsphereCredentials`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/auth/model/MindsphereCredentials.html). These objects will be created and shared between client instances. A builder pattern is used to instantiate them. ### Client Configuration The [`RestClientConfig`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/core/RestClientConfig.html) object can be built using configuration parameters. Currently all parameters are optional. ```java RestClientConfig config = RestClientConfig.builder() .connectionTimeoutInSeconds(100) .proxyHost("host") .proxyPort(8080) .build(); ``` The following parameters can be configured for the [`RestClientConfig`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/core/RestClientConfig.html) object: | Name | Description | Type | Default value | | -------------------------- | ------------------------------ | ------- | ------------- | | connectionTimeoutInSeconds | Connection timeout in seconds | Integer | 100 | | socketTimeoutInSeconds | Socket timeout in seconds | Integer | 100 | | proxyHost | Host address of proxy | String | | | proxyPort | Proxy port | Integer | | | proxyUsername | Username to login to the proxy | String | | | proxyPassword | Password to login to the proxy | String | | | hostEnvironment | Current Region | String | eu1 | | proxySchema | Schema used by the proxy | String | http | ### Credentials Configuration The [`MindsphereCredentials`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/auth/model/MindsphereCredentials.html) object can be built directly with the user token or with the technical token credentials. If credentials are set via environment variables, there is no need to build the [`MindsphereCredentials`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/auth/model/MindsphereCredentials.html) object. A code sample with user authorization token: ```java MindsphereCredentials credentials = MindsphereCredentials.builder() .authorization("usertokenFromRequestHeader") .build(); ``` A code sample with service credentials: ```java MindsphereCredentials credentials = MindsphereCredentials.builder() .clientId("ClientId") .clientSecret("ClientSecret") .tenant("TenantName") .build(); ``` The following parameters can be configured for the [`MindsphereCredentials`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/auth/model/MindsphereCredentials.html) object: | Name | Description | Type | Mandatory | | ------------- | ------------------------------------------------------------ | ------ | ----------------------------------------- | | authorization | Bearer token, if developer application already generated it. | String | Required for user token | | clientId | Service credential Id | String | Required for user with service credential | | clientSecret | Service credential | String | Required for user with service credential | | tenant | Name of the developer tenant | String | Required for user with service credential | ### API client instantiation and usage An API client instance requires a [`RestClientConfig`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/core/RestClientConfig.html) and a [`MindsphereCredentials`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/auth/model/MindsphereCredentials.html) instance passed as parameters using a builder pattern. A code sample using the IoT Time Series API client: ```java @RequestMapping(method = RequestMethod.GET, value = "/{entity}/{propertySetName}") public TimeseriesData getTimeSeriesAsObject(@PathVariable("entity") String entity, @PathVariable("propertySetName") String propertySetName, @RequestHeader("Authorization") String token) throws MindsphereException { MindsphereCredentials credentials = MindsphereCredentials.builder() .authorization(token) .build(); RestClientConfig config = RestClientConfig.builder() .connectionTimeoutInSeconds(100) .proxyHost("host") .proxyPort(portnumber) .hostEnvironment("host-environment") .build(); TimeseriesClient timeseriesClient = TimeseriesClient.builder() .mindsphereCredentials(credentials) .restClientConfig(config) .build(); TimeseriesData timeseriesData = null; try { timeseriesData = timeseriesClient.getLatestTimeseries(entity, propertySetName); } catch (MindsphereException e) { // Exception handling } return timeseriesData; } ``` # Asset Management Client for Java ## Introduction The Asset Management Java client allows you to interact with the digital representation of a machine or automation system through a Java object model, abstracting the interaction with the Industrial IoT API. Refer to [Asset Management](../../../apis/advanced-assetmanagement/api-assetmanagement-overview.html) for more information about the service. ## Aspect Type Operations The Aspect Type client manages static and dynamic aspect types. It lists, creates, updates and reads the aspect types. Client name: [AspectTypeClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/iot/asset/apiclient/AspectTypeClient.html) ### List all Aspect Types Get all aspect types of the tenant. ```java // Construct the AspectTypeClient object AspectTypeClient aspecttypeClient = AspectTypeClient.builder() .mindsphereCredentials(credentials) .restClientConfig(config) .build(); AspectTypes aspectTypes = null; try { aspectTypes = aspecttypeClient.getAspectTypes(page, size, sort, filter, ifNoneMatch); }catch (MindsphereException e) { // Exception handling } ``` ### Create an Aspect Type Create an aspect type. Aspect types can be defined to contain variables. ```java //Create client object "aspecttypeClient" as shown above AspectTypeResource aspectType = null; try { aspectType = aspecttypeClient.createAspectType(id, aspectTypeDto); } catch (MindsphereException e) { // Exception handling } ``` ### Update an Aspect Type Update an existing aspect type. Aspect types can be defined to contain variables. ```java //Create client object "aspecttypeClient" as shown above AspectTypeResource aspectType = null; try { aspectType = aspecttypeClient.updateAspectType(id, aspectTypeDto, IfMatch); } catch (MindsphereException e) { // Exception handling } ``` ### Read an Aspect Type Read an aspect type. ```java //Create client object "aspecttypeClient" as shown above AspectTypeResource aspectTypeResource = null; try { aspectTypeResource = aspecttypeClient.getAspectTypeById(id, ifNoneMatch); } catch (MindsphereException e) { // Exception handling } ``` ### Delete an Aspect Type Delete an aspect type. An aspect type can only be deleted if there is no asset type using it. ```java //Create client object "aspecttypeClient" as shown above boolean deleted = false; try { deleted = aspecttypeClient.deleteAspectType(id, ifMatch); } catch (MindsphereException e) { // Exception handling } ``` ## Asset Type Operations The Asset type client manages asset types. Client name: [AssetTypeClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/iot/asset/apiclient/AssetTypeClient.html) ### List all Asset Types List all asset types of the tenant. ```java // Construct the AssetTypeClient object AssetTypeClient assetTypeClient = AssetTypeClient.builder() .mindsphereCredentials(credentials) .restClientConfig(config) .build(); AssetTypes assetTypes = null; try { assetTypes = assetTypeClient.getAssetTypes(page, size, sort, filter, ifNoneMatch); } catch (MindsphereException e) { // Exception handling } ``` ### Create an Asset Type ```java //Create client object "assetTypeClient" as shown above AssetTypeResource assetTypeResource = null; AssetTypeDto assetTypeDto = new AssetTypeDto(); assetTypeDto.setParentTypeId(assetTypeId); assetTypeDto.setName(assetTypeName); assetTypeDto.setScope(assetTypeScope); try { assetTypeResource = assetTypeClient.createAssetType(id, assetTypeDto); } catch (MindsphereException e) { // Exception handling } ``` Aspects can be added to an asset type in the following ways: 1. Using Aspect Type name and Aspect Type id ```java //Create Asset Type Dto instance AssetTypeDto assetTypeDto = new AssetTypeDto(); assetTypeDto.setParentTypeId(assetTypeId); assetTypeDto.setName(assetTypeName); assetTypeDto.setScope(assetTypeScope); //Add the Aspect Type name and Aspect Type id to AssetTypeDto instance assetTypeDTO.addAspectsItem(aspectTypeName, aspectTypeId); ``` 1. Using AspectTypeResource instance ```java //Create Asset Type Dto instance as shown above //Create AspectTypeResource instance AspectTypeResource aspectTypeResource = new AspectTypeResource(); aspectTypeResource.setName(aspectTypeName); aspectTypeResource.setId(aspectTypeId); //Add the AspectTypeResource instance to AssetTypeDto instance assetTypeDTO.addAspectsItem(aspect); ``` 1. Using AssetTypeDtoAspects instance ```java //Create Asset Type Dto instance as shown above //Create AssetTypeDtoAspects instance AssetTypeDtoAspects assetTypeDtoAspects = new AssetTypeDtoAspects(aspectTypeName, aspectTypeId); //Add the AssetTypeDtoAspects instance to AssetTypeDto instance assetTypeDTO.addAspectsItem(assetTypeDtoAspects); ``` 1. Using a list of AspectTypeResource instances ```java //Create AssetTypeDto instance as shown above //Create AspectTypeResource instance AspectTypeResource aspectTypeResource = new AspectTypeResource(); aspectTypeResource.setName(aspectTypeName); aspectTypeResource.setId(aspectTypeId); //Create a List of AspectTypeResource instances and add the AspectTypeResource instance List aspects = new ArrayList<>(); aspects.add(aspectTypeResource); //Add the List of AspectTypeResource instances to AssetTypeDto instance assetTypeDTO.addAspectsItem(aspects); ``` 1. Using a list of AssetTypeDtoAspects instances ```java //Create AssetTypeDto instance as shown above //Create AssetTypeDtoAspects instance AssetTypeDtoAspects assetTypeDtoAspects = new AssetTypeDtoAspects(aspectTypeName, aspectTypeId); //Create a List of AssetTypeDtoAspects instances and add the AssetTypeDtoAspects instance List aspects = new ArrayList<>(); aspects.add(assetTypeDtoAspects); //Add the List of AssetTypeDtoAspects instances to AssetTypeDto instance assetTypeDTO.addAspectsItem(aspects); ``` ### Update an Asset Type ```java //Create client object "assetTypeClient" as shown above AssetTypeResource assetTypeResource = null; try { assetTypeResource = assetTypeClient.updateAssetType(id, assetTypeDto, ifMatch); } catch (MindsphereException e) { // Exception handling } ``` ### Read an Asset Type ```java //Create client object "assetTypeClient" as shown above AssetTypeResource assetTypeResource = null; try { assetTypeResource = assetTypeClient.getAssetTypeById(id, ifNoneMatch, exploded); } catch (MindsphereException e) { // Exception handling } ``` ### Delete an Asset Type Delete an asset type. Deletion is only possible when the type has no children and there is no asset that instantiates it. ```java //Create client object "assetTypeClient" as shown above boolean deleted = false; try { deleted = assetTypeClient.deleteAssetType(id, ifMatch); } catch (MindsphereException e) { // Exception handling } ``` ### Assign a File to an Asset Type ```java //Create client object "assetTypeClient" as shown above //Create FileAssignmentDto instance with required parameters. FileAssignmentDto fileAssignment = new FileAssignmentDto(); assignment.setFileId("fileId"); // Invoke the AssetTypeClient#updateFileAssignment method to assign a file to the asset type AssetTypeResource assetTypeResource = null; try { assetTypeResource = assetTypeClient.updateFileAssignment(assetTypeId, fileName, fileAssignment, ifMatch); } catch (MindsphereException e) { // Exception handling } ``` ### Delete a File from an Asset Type ```java //Create client object "assetTypeClient" as shown above // Invoke the AssetTypeClient#deleteFileAssignment method to remove a file from the asset type AssetTypeResource assetTypeResource = null; try { assetTypeResource = assetTypeClient.deleteFileAssignment(assetTypeId, fileName, ifMatch); } catch (MindsphereException e) { // Exception handling } ``` ## Asset Operations The Asset client manages the users' assets and their locations. Three different types of assets can be created: device types, agent types and hierarchy types. Client name: [AssetClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/iot/asset/apiclient/AssetClient.html) ### List all Asset Types List all assets available for the authenticated user. ```java // Construct the AssetClient object AssetClient assetClient = AssetClient.builder() .mindsphereCredentials(credentials) .restClientConfig(config) .build(); Assets assets = null; try{ assets = assetClient.getAssets(page, size, sort, filter, ifNoneMatch); } catch (MindsphereException e) { // Exception handling } ``` ### Create an Asset Type Create a new asset with the provided content. ```java //Create client object "assetClient" as shown above AssetResource assetResource = null; try{ assetResource = assetClient.createAsset(assetDto); } catch (MindsphereException e) { // Exception handling } ``` ### Read an Asset Read an asset. All static properties of the asset are returned. ```java //Create client object "assetClient" as shown above AssetResource assetResource = null; try{ assetResource = assetClient.getAssetById(id, ifNoneMatch); } catch (MindsphereException e) { // Exception handling } ``` ### Update an Asset Update an existing asset with the provided content. Only values can be modified, but not the asset structure. The asset structure can be modified in the asset type. ```java //Create client object "assetClient" as shown above AssetResource assetResource = null; try{ AssetResource assetResource = assetClient.updateAsset(id, assetDto, ifMatch); } catch (MindsphereException e) { // Exception handling } ``` ### Delete an Asset Delete an existing asset. After deletion, only the users with an admin role can read it, but modification is not possible anymore. It's not possible to delete an asset if it has children. ```java //Create client object "assetClient" as shown above Boolean deleted = false; try{ deleted = assetClient.deleteAsset(id, ifMatch); } catch (MindsphereException e) { // Exception handling } ``` ### Move an Asset Move an existing asset and all its children in the instance hierarchy. ```java //Create client object "assetClient" as shown above AssetResource assetResource = null; try{ assetResource = assetClient.moveAsset(id, assetMoveDto, ifMatch); } catch (MindsphereException e) { // Exception handling } ``` ### Read the Root Asset of the User Read the root asset of the user, from which the whole asset hierarchy can be rebuilt. ```java //Create client object "assetClient" as shown above AssetResource assetResource = null; try{ assetResource = assetClient.getRootAsset(ifNoneMatch); } catch (MindsphereException e) { // Exception handling } ``` ### Assign a File to an Asset ```java //Create client object "assetClient" as shown above //Create FileAssignmentDto instance with required parameters. FileAssignmentDto fileAssignment = new FileAssignmentDto(); assignment.setFileId("fileId"); // Invoke the AssetClient#updateFileAssignment method to assign a file to the asset AssetResource assetResource = null; try { assetResource = assetClient.updateFileAssignment(assetId, fileName, fileAssignment, ifMatch); } catch (MindsphereException e) { // Exception handling } ``` ### Delete a File from an Asset ```java //Create client object "assetClient" as shown above // Invoke the AssetClient#deleteFileAssignment method to remove a file from the asset AssetResource assetResource = null; try { assetResource = assetClient.deleteFileAssignment(assetId, fileName, ifMatch); } catch (MindsphereException e) { // Exception handling } ``` ## Asset Structure Operations Client name: [AssetStructureClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/iot/asset/apiclient/AssetStructureClient.html) ### Read Variables of an Asset Get all variables of an asset. ```java // Construct the AssetStructureClient object AssetStructureClient assetStructureClient = AssetStructureClient.builder() .mindsphereCredentials(credentials) .restClientConfig(config) .build(); StructureVariables structureVariables = null; try{ structureVariables = assetStructureClient.getVariables(id, page, size, sort, filter, ifNoneMatch); } catch (MindsphereException e) { // Exception handling } ``` ### Read all Aspects of an Asset Get all static and dynamic aspects of an asset. ```java //Create client object "assetStructureClient" as shown above StructureAspects structureAspects = null; try{ structureAspects = assetStructureClient.getAspects(id, page, size, sort, filter, ifNoneMatch); } catch (MindsphereException e) { // Exception handling } ``` ## Asset Location Operations Client name: [AssetLocationClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/iot/asset/apiclient/AssetLocationClient.html) ### Create or Update Location Creates or updates the location that is assigned to an asset. ```java // Construct the AssetLocationClient object AssetLocationClient assetLocationClient = AssetLocationClient.builder() .mindsphereCredentials(credentials) .restClientConfig(config) .build(); try{ assetLocationClient.createOrUpdateAssetLocation(id, location, ifMatch); } catch (MindsphereException e) { // Exception handling } ``` ### Delete Location assigned to the Asset ```java //Create client object "assetLocationClient" as shown above AssetResource assetResource = null; try{ assetResource = assetLocationClient.deleteAssetLocation(id, ifMatch); } catch (MindsphereException e) { // Exception handling } ``` ## Asset File Operations Client name: [AssetFileClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/iot/asset/apiclient/AssetFileClient.html) ### Upload a File Upload files to be associated with asset types or assets. ```java // Construct the AssetFileClient instance AssetFileClient assetFileClient = AssetFileClient.builder() .mindsphereCredentials(credentials) .restClientConfig(config) .build(); FileMetadataResource resource = null; try{ resource = assetFileClient.uploadAssetFiles(file, fileName, description); } catch (MindsphereException e) { // Exception handling } ``` ### List all Files List the metadata of all uploaded files. ```java // Construct the AssetFileClient instance as shown above AssetFiles assetFiles = null; try{ assetFiles = assetFileClient.getAssetFiles(); } catch (MindsphereException e) { // Exception handling } ``` ### Read the Metadata of a File Read the metadata of a file with the user defined ID. ```java // Construct the AssetFileClient instance as shown above AssetFiles assetFiles = null; try{ assetFiles = assetFileClient.getFileMetadataById(fileId, ifNoneMatch); } catch (MindsphereException e) { // Exception handling } ``` ### Read the Content of a File Read the content of a file with the user defined ID. ```java // Construct the AssetFileClient instance as shown above String fileContents = null; try{ fileContents = assetFileClient.getFileContentById(fileId, ifNoneMatch); } catch (MindsphereException e) { // Exception handling } ``` ## Page Iterators The page iterators for the Asset Management API client can be used to get a specific page, the next page or the previous page of data. The page iterators currently support the following operations: - getPage(Integer pageNumber); - next(); - previous(); The following iterators are provided for different controllers in Asset management: - [AspectTypePageIterator](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/iot/asset/iterator/model/AspectTypePageIterator.html) - [AssetTypePageIterator](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/iot/asset/iterator/model/AssetTypePageIterator.html) - [AssetPageIterator](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/iot/asset/iterator/model/AssetPageIterator.html) - [StructureAspectsPageIterator](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/iot/asset/iterator/model/StructureAspectsPageIterator.html) - [StructureVariablesPageIterator](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/iot/asset/iterator/model/StructureVariablesPageIterator.html) ```java try { // Create Asset Client AssetClient assetClient = AssetClient.builder() .mindsphereCredentials(credentials) .restClientConfig(config) .build(); // Create Asset Page Iterator AssetPageIterator assetPageIterator = new AssetPageIterator(assetClient, pageSize); Assets assets = assetPageIterator.getPage(10); } catch (MindsphereException e) { // Exception handling } ``` # Event Analytics Client for Java ## Introduction The Event Analytics Java client allows you to analyze event data. In addition to the identification of significant dependencies, statistical analysis also helps the user to get a better understanding of the system's internal processes. ## Event Analytics Operations The Event Analytics statistically analyzes the event data to identify the most frequent events. Client name: [EventAnalyticsClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/analytics/apiclient/EventAnalyticsClient.html) ### Find Top Events This method finds the N most frequently occurring events and returns them sorted by the number of occurrences in descending order. ```java // Creating the Event Analytics Input Metadata object EventAnalyticsRequestMetadata inputMetadata = new EventAnalyticsRequestMetadata(); inputMetadata.setEventTextPropertyName("text"); // Creating a list of events to be analytically Event firstEvent = new Event(); firstEvent.setText("INTRODUCING FUEL"); firstEvent.setTextQc(0); firstEvent.setTime("2017-10-01T12:00:00.001Z"); Event secondEvent = new Event(); secondEvent.setText("Status@Flame On"); secondEvent.setTextQc(0); secondEvent.setTime("2017-10-01T12:02:01.001Z"); List eventsList = new ArrayList<>(); eventsList.add(firstEvent); eventsList.add(secondEvent); // Creating the Event Analytics Input object that will be analyzed EventAnalyticsRequest eventAnalyticsRequest = new EventAnalyticsRequest(); eventAnalyticsRequest.setNumberOfTopPositionsRequired(5); eventAnalyticsRequest.setEventsMetadata(inputMetadata); eventAnalyticsRequest.setEvents(eventsList); List eventAnalyticsResponse = null; try { eventAnalyticsResponse = eventAnalyticsClient.findTopEvents(eventAnalyticsRequest); } catch (MindsphereException e) { // Exception handling } ``` # Event Management Client for Java ## Introduction The Event Management Java client allows you to create and update events associated with assets. It also manages different types of events created by users. Refer to [Event Management](../../../apis/advanced-eventmanagement/api-eventmanagement-overview.html) for more information about the service. ## Event Operations The Event Controller manages standard and custom events. It creates, lists, and updates the events based on the API calls. Client name: [EventClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/iot/event/apiclient/EventClient.html) ### Create a New Event Note Entity IDs are cached with 1 minute retention. When an entity (asset) is created, it might only be visible for Event Management after the cache is refreshed. ```java // Construct the EventClient object EventClient eventClient = EventClient.builder() .mindsphereCredentials(credentials) .restClientConfig(config) .build(); // Json string representing Mindsphere standard event or custom event String eventJsonString = "{\r\n" + " \"typeId\": \"3cb8c2b1-f72e-43fa-ae39-ee9de14b8f26\",\r\n" + " \"correlationId\": \"3ecc3658-96bb-47ff-93b3-93afd640ee10\",\r\n" + " \"entityId\": \"50fd8ab5-23b3-45f7-aca1-12b0b186707b\",\r\n" + " \"timestamp\": \"2018-01-09T13:40:23.438Z\",\r\n" + " \"customStringField\": \"customValue\",\r\n" + " \"customIntegerField\": 10\r\n" + "}" BaseEvent createdEvent = null; try { createdEvent = eventClient.createEvent(eventJsonString); } catch (MindsphereException ex) { // Exception Handling } ``` ### Create a New Custom Event A custom event is an event, which has been assigned to a user-defined event type. ```java // Construct the EventClient object as shown above //Construct the CustomEvent object to be created CustomEvent customEvent = new CustomEvent(); customEvent.setTypeId(eventTypeId); customEvent.setEntityId(entityId); customEvent.setTimestamp(timestamp); Map fields = new HashMap<>(); fields.put("customField", "customValue"); customEvent.setFields(fields); CustomEvent createdCustomEvent = null; try { createdCustomEvent = eventClient.createCustomEvent(customEvent); } catch (MindsphereException ex) { // Exception Handling } ``` ### Create a New Standard Event Standard events use a predefined event type with the following additional properties: - description - severity - code - source - acknowledged ```java // Construct the EventClient object as shown above //Construct the StandardEvent object to be created MindsphereStandardEvent standardEvent = new MindsphereStandardEvent(); standardEvent.setDescription(description); standardEvent.setSeverity(severity); standardEvent.setCode(code); standardEvent.setAcknowledged(acknowledged); standardEvent.setEntityId(entityId); standardEvent.setTimestamp(timestamp); MindsphereStandardEvent createdStandardEvent = null; try { createdStandardEvent = eventClient.createStandardEvent(standardEvent); } catch (MindsphereException ex) { // Exception Handling } ``` ### Query Events Returns events based on the query parameters. Note The default number of elements that will be returned in a page is 20. ```java // Construct the EventClient object as shown above Events events = null; try { events = eventClient.getEvents(size, page, filter, sort, ifNoneMatch, history); } catch (MindsphereException ex) { // Exception Handling } ``` ### Query an Event by ID Returns an event with the specified ID. ```java // Construct the EventClient object as shown above BaseEvent event = null; try { event = eventClient.getEventById(eventId); // The retrieved event can be checked for type as follows if (event != null && event instanceof MindsphereStandardEvent) { // Process the retrieved Standard event } else if (event != null && event instanceof CustomEvent) { // Process the retrieved custom event } } catch (MindsphereException ex) { // Exception Handling } ``` ### Update an Event Updates an existing event with the specified ID. ```java // Construct the EventClient object as shown above BaseEvent event = null; try { event = eventClient.updateEvents(eventId, eventJsonString, ifMatch); // The retrieved event can be checked for type as shown above for further processing } catch (MindsphereException ex) { // Exception Handling } ``` ### Update a Standard Event Updates an existing standard event with the specified ID. ```java // Construct the EventClient object as shown above MindsphereStandardEvent updatedStandardEvent = null; try { updatedStandardEvent = eventClient.updateEvents(eventId, standardEvent, ifMatch); } catch (MindsphereException ex) { // Exception Handling } ``` ### Update a Custom Event Updates an existing custom event with the specified ID. ```java // Construct the EventClient object as shown above CustomEvent customEvent = null; try { customEvent = eventClient.updateEvents(eventId, customEvent, ifMatch); } catch (MindsphereException ex) { // Exception Handling } ``` ## Event Type Operations Event type operations create, list, and update the event types based on the API calls. Client name: [EventTypeClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/iot/event/apiclient/EventTypeClient.html) ### Create a New Event Type Creates a new event type with the specified content. The Default ID is the generated UUID. If a custom ID is provided, it has to be prefixed with the tenant name followed by a dot. ```java // Construct the EventTypeClient object EventTypeClient eventTypeClient = EventTypeClient.builder() .mindsphereCredentials(credentials) .restClientConfig(config) .build(); // Define the Event Type to be created String eventTypeName = "OilTemperatureRise"; EventType eventType = new EventType(); eventType.setName(eventTypeName); eventType.setTtl(10); eventType.setScope(ScopeEnum.LOCAL); // Define Field values to be associated with the Event Type Field field = new Field(); field.setName("temperature"); field.setFilterable(Boolean.TRUE); field.required(Boolean.TRUE); field.setType(FieldTypeEnum.INTEGER); // Associate Field values to the Event Type List fieldList = new ArrayList<>(); fieldList.add(field); eventType.setFields(fieldList); EventType createdEventType = null; try { createdEventType = eventTypeClient.createEventType(eventType); } catch (MindsphereException e) { // Exception handling } ``` ### Query Event Types Returns event types based on the query parameters. ```java //Construct EventTypeClient object as shown above EventTypes eventTypes = null; try { eventTypes = eventTypeClient.getEventTypes(page, size, sort, filter, ifNoneMatch); } catch (MindsphereException e) { // Exception Handling } ``` ### Read an Event Type by ID Returns an event type by ID. ```java //Construct EventTypeClient object as shown above EventType eventType = null; try { eventType = eventTypeClient.getEventTypesById(eventTypeId, ifNoneMatch); } catch (MindsphereException e) { // Exception Handling } ``` ### Update an Event Type Update the event type with the specified ID by sending only the content to be changed. ```java //Construct EventTypeClient object as shown above //Define the Event Tyoe Patch that will be used to update the event Type EventTypePatch eventTypePatch = new EventTypePatch(); eventTypePatch.setOperation(OperationEnum.REPLACE); eventTypePatch.setPath("/scope"); eventTypePatch.setValue(ScopeEnum.GLOBAL.getValue()); EventType updatedEventType = null; try { updatedEventType = eventTypeClient.updateEventType(eventTypeId, eventType.getEtag().toString(), patch); } catch (MindsphereException e) { // Exception Handling } ``` ## Event Job Operations The Event Job Controller can be used to create jobs that delete queried events. Client name: [EventJobClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/iot/event/apiclient/EventJobClient.html) ### Create a Job to Delete Events Create a new job to delete events, which satisfy query parameters. The query parameters for selecting the events to be deleted are mentioned as a filter in the [DeleteEventsJobDto](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/iot/event/model/DeleteEventsJobDto.html) instance. ```java // Construct the EventJobClient object EventJobClient eventJobClient = EventJobClient.builder() .mindsphereCredentials(credentials) .restClientConfig(config) .build(); //Create a DeleteEventsJobDto instance DeleteEventsJobDto deleteEventsJobDto = new DeleteEventsJobDto(); deleteEventsJobDto.setFilter("typeId", "eventTypeId"); JobResource jobResource = null; try { jobResource = eventJobClient.createDeleteEventsJob(deleteEventsJobDto); } catch (MindsphereException e) { // Exception handling } ``` DeleteEventsJobDto instance can be populated with criteria in the following ways: 1. Using filter key and value ```java DeleteEventsJobDto deleteEventsJobDto = new DeleteEventsJobDto(); deleteEventsJobDto.setFilter("typeId", "eventTypeId"); deleteEventsJobDto.setFilter("id", "eventId"); ``` 1. Using a HashMap ```java DeleteEventsJobDto deleteEventsJobDto = new DeleteEventsJobDto(); HashMap filterMap = new HashMap<>(); filterMap.put("typeId", "eventTypeId"); filterMap.put("id", "eventId"); deleteEventsJobDto.setFilter(filterMap); ``` 1. Using filter a string ```java DeleteEventsJobDto deleteEventsJobDto = new DeleteEventsJobDto(); String filter = "{\"filter\": { \"id\":\"eventId\"," + \"typeId\":\"eventTypeId\"\r\n" + " } " + "}"; deleteEventsJobDto.setFilter(filter); ``` ### Read a Job to Delete Events by ID Returns a job to delete events by ID. ```java //Construct EventJobClient object as shown above String jobId = "deleteEventsJobId"; JobResource jobResource = null; try { jobResource = eventJobClient.getDeleteEventsJobById(deleteEventsJobId); } catch (MindsphereException e) { // Exception handling } ``` # IoT File Client for Java ## Introduction The IoT File Java client allows you to manage files related to assets. Refer to [IoT File Service](../../../apis/iot-iotfile/api-iotfile-overview.html) for more information about the service. Hint In the IoT context, assets are referred to as `entity` and aspects as `propertyset`. ## File Operations Client name: [FileservicesClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/iot/fileservice/apiclient/FileservicesClient.html) ### Create a File Create a file for the specified entity and path with the provided content. ```java FileservicesClient fileservicesClient = FileservicesClient.builder() .mindsphereCredentials(credentials) .restClientConfig(config) .build(); FileWriterResponse fileWriterResponse; try { fileWriterResponse = fileservicesClient.createFile(file,entityId,filepath,description,type); } catch (MindsphereException e) { // Exception handling } ``` ### Update a File Update a file for the specified entity and path with the provided content. ```java //Construct FileservicesClient object as shown above FileWriterResponse fileWriterResponse; try { fileWriterResponse = fileservicesClient.updateFile(file, entityId, filepath, description, type, ifMatch); } catch (MindsphereException e) { // Exception handling } ``` ### Read a File Read a file for the specified entity and path. The `GET` method is used for the API call. ```java //Construct FileservicesClient object as shown above FileReaderResponse fileReaderResponse; try { fileReaderResponse = fileservicesClient.readFile(entityId, filepath, ifNoneMatch) } catch (MindsphereException e) { // Exception handling } ``` ### Delete a File Delete a file for the specified entity and path. ```java //Construct FileservicesClient object as shown above boolean isFileDeleted; try { isFileDeleted = fileservicesClient.deleteFile(entityId, filepath) } catch (MindsphereException e) { // Exception handling } ``` ### Search a File Search one or multiple files for the specified entity and path. ```java //Construct FileservicesClient object as shown above FileSearchResponse fileSearchResponse; try { fileSearchResponse = fileservicesClient.searchFiles(entityId, offset, limit, count, order, filter); } catch (MindsphereException e) { // Exception handling } ``` # IoT Time Series Client for Java ## Introduction The IoT Time Series Java client allows you to interact with time series data related to assets. Refer to [IoT Time Series](../../../apis/iot-iottimeseries/api-iottimeseries-overview.html) for more information about the service. Hint In the IoT context, assets are referred to as `entity` and aspects as `propertyset`. ## Time Series Operations Client Name: [TimeseriesClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/iot/timeseries/apiclient/TimeseriesClient.html) ### Read latest Time Series Data Read the latest value of time series data for a single entity and property set. ```java // Construct the TimeseriesClient object TimeseriesClient timeseriesClient = TimeseriesClient.builder() .mindsphereCredentials(credentials) .restClientConfig(config) .build(); TimeseriesData timeseriesData = null; try { timeseriesData = timeseriesClient.getLatestTimeseries(entity, propertySetName); } catch (MindsphereException e) { // Exception handling } ``` ### Read Time Series Data This operation invokes the IoT Time Series API to: - Read time series data for a single entity and property set. - Return data for a specified time range. - Return the latest value, if no range is provided. - Return time series data for selected fields and limit. ```java //Construct TimeseriesClient object as shown above List timeseriesDataList = null; try { timeseriesDataList = timeseriesClient.getTimeseries(entity, propertySetName, from, to, limit, select); } catch (MindsphereException e) { // Exception handling } ``` ### Write Time Series Data Write the time series for a single entity and property set. The existing time series data will be overwritten. Note The maximum size of a write payload is 1 MB. ```java //Construct TimeseriesClient object as shown above boolean success = false; try { success = timeseriesClient.putTimeSeries(entity, propertySetName, timeSeriesList, sync); } catch (MindsphereException e) { // Exception handling } ``` ### Import Time Series Data Import or update time series for a single entity and property set. The existing time series data is overwritten. ```java //Construct TimeseriesClient object as shown above boolean success = false; try { success = timeseriesClient.importTimeseries(entity, propertySetName, timeseriesFile, sync); } catch (MindsphereException e) { // Exception handling } ``` ### Delete Time Series Data Delete time series entity and property set within a given time range. It also deletes data for all properties within a property set. Note Only data of up to 6 months can be deleted at once. ```java //Construct TimeseriesClient object as shown above boolean success = false; try { success = timeseriesClient.deleteTimeSeries(entity, propertySetName, from, to, sync); } catch (MindsphereException e) { // Exception handling } ``` ## Page Iterators The page iterator for the IoT Time Series API client, [TimeseriesPageIterator](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/iot/timeseries/iterator/model/TimeseriesPageIterator.html) can be used to get a specific page, the next page or the previous page of data. The page iterators currently support the following operations: - `getPage(Integer pageNumber);` - `next();` - `previous();` The page iterator instantiation requires an IoT Time Series API client, an entity, a property set name, a date range (`from` and `to` dates) and a page size. If the page size is not provided, it defaults to 2000. Code sample for the IoT Time Series page iterator instantiation and usage: ```java try { //Create the timeseries client TimeseriesClient timeseriesClient = TimeseriesClient.builder() .mindsphereCredentials(credentials) .restClientConfig(config) .build(); // Create the timeseries page iterator TimeseriesPageIterator timeseriesPageIterator = new TimeseriesPageIterator(timeseriesClient, entityId, propertySetName, fromDateString, toDateString, pageSize); List timseriesDataList = timeseriesPageIterator.getPage(10); } catch (MindsphereException e) { // Exception handling } ``` # IoT TS Aggregates Client for Java ## Introduction The IoT TS Aggregates Java client allows you to query aggregated time series data. Refer to [IoT TS Aggregates](../../../apis/iot-iottsaggregates/api-iottsaggregates-overview.html) for more information about the service. Hint In the IoT context, assets are referred to as `entity` and aspects as `propertyset`. ## TS Aggregates Operations Client name: [TimeseriesAggregateClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/iot/aggregate/apiclient/TimeseriesAggregateClient.html) ### Read Aggregated Time Series Data Read aggregated time series data. This method returns aggregated data of a single aspect of an asset from the specified time range. Note A query can only return up to 200 aggregate intervals in one response.\ Aggregates can only be created from up to 5,000 raw time series entries. ```java // Construct the TimeseriesAggregateClient object TimeseriesAggregateClient timeseriesAggregateClient = TimeseriesAggregateClient.builder() .mindsphereCredentials(credentials) .restClientConfig(config) .build(); List aggregateDetailsList = null; try { aggregateDetailsList = timeseriesAggregateClient.getAggregateTimeseries(entity, propertySet, from, to, intervalValue, intervalUnit, select); } catch (MindsphereException e) { // Exception handling } ``` # Industrial IoT SDK for Java - API clients and References The complete Javadoc documentation for the Industrial IoT SDK for Java in version 1.2.2 can be found [**here**](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/index.html). Below is an overview of the Industrial IoT SDK for Java client implementations grouped by service with links to the Javadoc documentation: ## Asset Management - [AspectTypeClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/iot/asset/apiclient/AspectTypeClient.html) - [AssetTypeClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/iot/asset/apiclient/AssetTypeClient.html) - [AssetClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/iot/asset/apiclient/AssetClient.html) - [AssetStructureClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/iot/asset/apiclient/AssetStructureClient.html) - [AssetLocationClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/iot/asset/apiclient/AssetLocationClient.html) - [AssetFileClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/iot/asset/apiclient/AssetFileClient.html) ### Page Iterators - [AspectTypePageIterator](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/iot/asset/iterator/model/AspectTypePageIterator.html) - [AssetTypePageIterator](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/iot/asset/iterator/model/AssetTypePageIterator.html) - [AssetPageIterator](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/iot/asset/iterator/model/AssetPageIterator.html) - [StructureAspectsPageIterator](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/iot/asset/iterator/model/StructureAspectsPageIterator.html) - [StructureVariablesPageIterator](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/iot/asset/iterator/model/StructureVariablesPageIterator.html) ## IOT Time Series - [TimeseriesClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/iot/timeseries/apiclient/TimeseriesClient.html) ### Page Iterator - [TimeseriesPageIterator](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/iot/timeseries/iterator/model/TimeseriesPageIterator.html) ## IoT TS Aggregates - [TimeseriesAggregateClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/iot/aggregate/apiclient/TimeseriesAggregateClient.html) ## IoT File - [FileservicesClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/iot/fileservice/apiclient/FileservicesClient.html) ## Event Management - [EventClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/iot/event/apiclient/EventClient.html) - [EventTypeClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/iot/event/apiclient/EventTypeClient.html) - [EventJobClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/iot/event/apiclient/EventJobClient.html) ## Event Analytics - [EventAnalyticsClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java/javadocs/html/com/siemens/mindsphere/sdk/analytics/apiclient/EventAnalyticsClient.html) # Industrial IoT SDK V2 for Java ## Introduction The Industrial SDK V2 for Java enables developers to work with Industrial IoT easily. It is divided into a core module and several service modules, which allows developers to include only those parts of the SDK required for their use case. The core module provides authentication, exception handling, logging features and a common Java client implementation for the RESTful Industrial IoT APIs. All service modules have a dependency on the core module. The Industrial IoT SDK V2 for Java provides client implementations for the following APIs: | Name | API Version | SDK Version | | ----------------------------------------------------- | ----------- | ----------- | | SDK Core | | 2.4.0 | | [Agent Management](apidocs/AgentManagement.html) | 3.2.1 | 3.5.2 | | [Asset Management](apidocs/AssetManagement.html) | 3.9.1 | 3.9.9 | | [Event Management](apidocs/EventManagement.html) | 3.8.1 | 3.8.10 | | [IoT File](apidocs/IOT_File.html) | 3.2.4 | 3.3.0 | | [IoT Time Series](apidocs/IOT_TimeseriesV.html) | 3.4.4 | 3.4.5 | | [IoT TS Aggregates](apidocs/TSAggregateV4.html) | 4.0.0 | 4.2.1 | | [Token Management Service](apidocs/TokenManager.html) | 3.0.0 | 3.1.5 | | [Integrated Data Lake](apidocs/IDL.html) | 3.2.0 | 3.2.4 | | [IoT Bulk Data](apidocs/IOT_Bulk.html) | 3.5.0 | 3.5.1 | | [MindConnect](apidocs/MindConnect.html) | 3.5.1 | 3.5.3 | Following older SDK & API versions can be referred for backward compatibility & will be deprecated in upcoming SDK releases: | Name | API Version | SDK Version | | ----------------------------------------------- | ----------- | ----------- | | [IoT Time Series](apidocs/IOT_Timeseries.html) | 3.4.1 | 3.4.1 | | [IoT TS Aggregates](apidocs/IOT_Aggregate.html) | 3.1.1 | 3.1.6 | Refer to the [Getting Started](sdkreadme_v2.html) for installation and configuration of an API client. ## Get the SDK The Industrial IoT SDK V2 for Java is available for download on the [Siemens Industry Online Support (SIOS) Portal](https://support.industry.siemens.com/cs/document/109757603). The SDK core and service specific modules are in JAR format and bundled into a ZIP file. It is possible to only unzip those modules you intend to use. Download for region China 1 For region China 1, download the zip file with `Region China` special label version from the Siemens Industry Online Support (SIOS) Portal. ## Features The Industrial IoT SDK V2 for Java provides the same core features as the previous version. It bundles these features in the core module, which is separate from the individual service modules. ### Industrial IoT SDK Core The core module of the Industrial IoT SDK V2 for Java handles authorization, client configuration, exception handling, and common API Client for the RESTful Industrial IoT APIs. All the clients have a dependency on the core module. #### Client Configuration You can easily configure the Industrial IoT SDK V2 for Java through configuration parameters such as proxy settings and connection time-outs. #### Token Handling Mechanism The Industrial IoT SDK V2 for Java provides an easy authorization handling mechanism. It handles technical token fetching, caching, validation and re-fetching based on user authorization tokens or service credentials configured by developers. Refer to [Token Handling](token_handling_v2.html) for more information. #### SDK Xcelerator Adoption The Industrial IoT SDK V2 for Java now supports Xcelerator domain resources, enabling applications to leverage Xcelerator features. For more information, refer to the [Xcelerator Adoption in SDK & Platform](xcelerator_sdk_adoption.html) section. #### Logging Logging is configured with SLF4J, which is an abstraction layer that enables the use of any logging framework at runtime, such as Java Logging Framework and Apache Log4j. The logging can be enabled or disabled by the developer application. Refer to [Logging](xcelerator_sdk_adoption.html) for more information. #### Exception Handling The Industrial IoT SDK V2 for Java provides an exception handling mechanism. Different exception classes help to identify whether an exception was caused by your client or by the Industrial IoT server. Detailed information such as error code, message, HTTP status, logref, and cause are provided for investigation. For more information,refer to [Exception Handling](exception_handling_v2.html). ### Page Iterators The Industrial IoT SDK V2 for Java provides page iterator objects for most API clients, which allows for easy pagination. Developer applications can configure paging parameters and request the next/previous page or a specific page number. # Industrial IoT SDK V2 for Java - Exception Handling Info Exception handling in the Industrial IoT SDK V2 for Java is identical to the one of V1. Understanding how and when the Industrial IoT SDK V2 for Java throws exceptions is important to build high-quality applications using the SDK. The following sections describe the different scenarios in which exceptions are thrown by the SDK and how to handle them appropriately. ## MindsphereException [`MindsphereException`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/core/exception/MindsphereException.html) is the most common exception that you'll experience when using the Industrial IoT SDK for Java. This exception represents an error response from an Industrial IoT service. MindsphereException provides you with information such as: - Returned HTTP status - Returned HTTP status Code - Detailed error message from the service - Reference to the occurred error ### Structure of MindsphereException | Name | Description | Type | | ----------------- | ------------------------------------- | ---------- | | `errorStatusCode` | Error code (if returned by API) | String | | `errorMessage` | Error message | String | | `httpStatus` | HTTP status | HttpStatus | | `logref` | Logger reference (if returned by API) | String | | `cause` | Cause of the exception (if available) | Throwable | ### Subclasses of MindsphereException | Subclass | Parent | Description | | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- | | [MindsphereServiceException](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/core/exception/MindsphereServiceException.html) | [MindsphereException](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/core/exception/MindsphereException.html) | Indicates an error at the API endpoint. For example, if the user tries to call an API with invalid values for mandatory parameters. | | [MindsphereClientException](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/core/exception/MindsphereClientException.html) | [MindsphereException](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/core/exception/MindsphereException.html) | Indicates an error due to missing parameter values in a request. For example, if the user tries to call an API with NULL values for mandatory parameters. | | [MindsphereClientConfigurationException](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/core/exception/MindsphereClientConfigurationException.html) | [MindsphereClientException](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/core/exception/MindsphereClientException.html) | Indicates an error due to incorrect configuration parameters. For example, if a user sets an incorrect value for the proxy host or proxy port. | | [MindsphereForbiddenAccessException](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/core/exception/MindsphereForbiddenAccessException.html) | [MindsphereClientException](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/core/exception/MindsphereClientException.html) | Indicates that no valid token can be retrieved with the provided credentials. | # Industrial IoT SDK V2 for Java - Logging Logging of the Industrial IoT SDK V2 for Java is implemented with SLF4J (Simple Logging Facade for Java), which provides access to many logging frameworks such as Log4j, java.util.logging. This allows the end user to utilize the desired logging framework. The Industrial IoT SDK V2 for Java uses the same logging mechanism as V1. For detailed documentation on how to use the logging functionality under different frameworks, refer to the [Logging Documentation of V1](../industrial-iot-sdk-java/sdk_logging.html). # Industrial IoT SDK V2 for Java – Getting Started ## Prerequisites to use the Industrial IoT SDK V2 for Java - Java 17 is available. - Gradle or Maven is used as build tool. - User authorization token or service credentials with required scopes for APIs are available. - The `HOST_ENVIRONMENT` environment variable is set to the current region. When hosting an application, the variable must be added in the manifest file: ```text env: HOST_ENVIRONMENT: eu1 ``` If not specified, `HOST_ENVIRONMENT` defaults to `eu1`. in region Europe 1 SDK and to `cn1` in region China 1 SDK.\ \* The `HOST_BASEDOMAIN` environment variable is optional and is required only if the domain is not `*.mindsphere.io`. When hosting an application, this variable must be added to the manifest file: ```text env: HOST_BASEDOMAIN: example.orgname.com ``` - The `MDSP_MANAGEMENT_TENANT` management tenant variable is optional and should be set only if your environment does not use the default production management tenant `1000001700` in upstream service URLs. When hosting the application, this variable must be added to the manifest file: ```text env: MDSP_MANAGEMENT_TENANT: managementtenantid ``` Hint Service credentials can be set as environment variables so the client can fetch a token itself. - `MINDSPHERE_CLIENT_ID`\ Specifies service credentials ID - `MINDSPHERE_CLIENT_SECRET`\ Specify service credentials secret - `MINDSPHERE_TENANT`\ Specifies the tenant name Attention You are responsible for keeping the credentials safe. You decide whether it is safe to supply the credentials via environment variables. ## Installation Instructions ### Downloading the Industrial IoT SDK V2 for Java [Download](index.html#get-the-sdk) the Industrial IoT SDK for Java from the Siemens Industry Online Support (SIOS) Portal. The `jar` and `pom` files of the core module and the service modules have the following structure: ```text com\siemens\mindsphere\mindsphere-sdk-java-core\{x.y.z}\mindsphere-sdk-java-core-{x.y.z}.jar com\siemens\mindsphere\mindsphere-sdk-java-core\{x.y.z}\mindsphere-sdk-java-core-javadoc.jar com\siemens\mindsphere\mindsphere-sdk-java-core\{x.y.z}\mindsphere-sdk-java-core.pom ``` ```text com\siemens\mindsphere\{service_name}-sdk\{x.y}\{service_name}-sdk-{x.y}.jar com\siemens\mindsphere\{service_name}-sdk\{x.y}\{service_name}-sdk-{x.y}-javadoc.jar com\siemens\mindsphere\{service_name}-sdk\{x.y}\{service_name}-sdk-{x.y}.pom ``` Note - `{x.y.z}` is the version number of the Industrial IoT Core SDK for Java (e.g. `2.0.0`).The file `{service_name}-sdk-{x.y.z}.pom` is required for downloading the transitive dependencies of the Industrial IoT Core SDK for Java. - `{x.y}` is the version number of the API Specification (e.g. `3.0`). The file `{service_name}-sdk-{x.y}.pom` is required for downloading the transitive dependencies of the Industrial IoT Service SDK for Java. ### Adding Industrial IoT SDK V2 Dependencies 1. Create the core and service module folder structure as your local Maven repository : ```text $PATH\com\siemens\mindsphere\mindsphere-sdk-java-core\{x.y.z}\mindsphere-sdk-java-core-{x.y.z}.jar $PATH\com\siemens\mindsphere\mindsphere-sdk-java-core\{x.y.z}\mindsphere-sdk-java-core-javadoc.jar $PATH\com\siemens\mindsphere\mindsphere-sdk-java-core\{x.y.z}\mindsphere-sdk-java-core.pom ``` ```text $PATH\com\siemens\mindsphere\{service_name}-sdk\{x.y}\{service_name}-sdk-{x.y}.jar $PATH\com\siemens\mindsphere\{service_name}-sdk\{x.y}\{service_name}-sdk-{x.y}-javadoc.jar $PATH\com\siemens\mindsphere\{service_name}-sdk\{x.y}\{service_name}-sdk-{x.y}.pom ``` where `$PATH` is: - On Mac: `~/.m2/repository` - On Windows: `C:\Users\{user_name}\.m2\repository` - On Linux: `/home/{user_name}/.m2/repository` 2.Register the repository and include dependencies in the build config file of your project. ```text maven-repository file:///{absolute_path_of_the_repository_folder} com.siemens.mindsphere mindsphere-sdk-java-core {x.y.z} com.siemens.mindsphere {service_name}-sdk {x.y} ``` ```javascript // Modify the following sections of the build.gradle file // 1. Register the repository in the repositories section repositories { maven { url file('{Absolute_path_of_the_created_repository_folder}') } mavenLocal() mavenCentral() } // 2. Add Core dependency in the dependencies section (optional) compile 'com.siemens.mindsphere:mindsphere-sdk-java-core:{x.y.z}' // 3. Add Service dependency in the dependencies section compile 'com.siemens.mindsphere:{service_name}-sdk:{x.y}' ``` Note All the Industrial IoT service modules have an implicit dependency on the core module, so adding the core module dependency is optional. Further implementation of the SDK libraries has been shown in a sample project that you can download and test in local or on Industrial IoT application. Please refer to this repository: [industrial-iot-java-sdk-examples](https://github.com/mindsphere/mindsphere-java-sdk-examples). ## API Client and Credentials Configuration The lowest-level building blocks of the API are [`RestClientConfig`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/core/RestClientConfig.html) and [`MindsphereCredentials`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/core/MindsphereCredentials.html). These objects are instantiated using a builder pattern and shared between client instances. ### Client Configuration The following code block shows an example of how to build a [`RestClientConfig`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/core/RestClientConfig.html) object: ```java RestClientConfig config = RestClientConfig.builder() .connectionTimeoutInSeconds(100) .proxyHost("host") .proxyPort(8080) .build(); ``` The [`RestClientConfig`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/core/RestClientConfig.html) can be configured using the following optional parameters: | Name | Description | Type | Default value | | ---------------------------- | ------------------------------ | ------- | ------------- | | `connectionTimeoutInSeconds` | Connection timeout in seconds | Integer | `100` | | `socketTimeoutInSeconds` | Socket timeout in seconds | Integer | `100` | | `proxyHost` | Host address of the proxy | String | | | `proxyPort` | Proxy port | Integer | | | `proxyUsername` | Username to login to the proxy | String | | | `proxyPassword` | Password to login to the proxy | String | | | `hostEnvironment` | Current Region | String | `eu1` | | `proxySchema` | Schema used by the proxy | String | `http` | ### Credentials Configuration The [`MindsphereCredentials`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/core/MindsphereCredentials.html) object can be built with a user token or with service credentials to fetch a technical token. The following configuration parameters are available for the [`MindsphereCredentials`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/core/MindsphereCredentials.html) object: | Name | Description | Type | Use Case | | ---------------------- | --------------------------------- | ------ | --------------- | | `authorization` | Bearer token, if available. | String | user token | | `keyStoreClientId` | App service credential ID | String | technical token | | `keyStoreClientSecret` | App service credential secret | String | technical token | | `appName` | Application name | String | technical token | | `appVersion` | Application version | String | technical token | | `hostTenant` | Host tenant | String | technical token | | `userTenant` | User tenant | String | technical token | | `clientId` | Tenant service credentials ID | String | technical token | | `clientSecret` | Tenant service credentials secret | String | technical token | | `tenant` | Tenant | String | technical token | Refer to the [code samples for creating Insights HubCredentials objects](apidocs/Credentials.html) for more information. ### API Client Instantiation and Usage An API client instance using [`RestClientConfig`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/core/RestClientConfig.html) and [`MindsphereCredentials`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/core/MindsphereCredentials.html) objects passed as (optional) parameters using a builder pattern. Code sample using the IoT TimeSeries API client, placeholders are indicated by angle brackets `< >`: ```java @RequestMapping(method = RequestMethod.GET, value = "/{entity}/{propertySetName}") public Timeseries getTimeSeriesAsObject(@PathVariable("entity") String entity, @PathVariable("propertySetName") String property_set_name, @RequestHeader("Authorization") String token) throws MindsphereException { //Construct MindsphereCredentials object MindsphereCredentials credentials = MindsphereCredentials.userTokenBuilder() .authorization(token) .build(); //Construct RestClientConfig object RestClientConfig config = RestClientConfig.builder() .connectionTimeoutInSeconds(100) .proxyHost("my.proxy.host") .proxyPort("1.2.3.4") .hostEnvironment("eu1") .build(); //Construct TimeSeriesClient object TimeSeriesClient timeseries_client = TimeSeriesClient.builder() .mindsphereCredentials(credentials) .restClientConfig(config) .build(); //Construct GetTimeseriesRequest object GetTimeseriesRequest request_object = new GetTimeseriesRequest(); request_object.setEntity(entity); request_object.setPropertysetname(propertySetName); request_object.setFrom(); request_object.setTo(); List timeseries_data_list = null; try { timeseries_data_list = timeseries_client.getTimeseries(request_object); } catch (MindsphereException e) { // Exception handling } return timeseries_data_list; } ``` # Industrial IoT SDK V2 for Java - Token Handling Token Handling in Industrial IoT SDK V2 for Java provides access token fetching using service credentials, caching them and re-fetching them on expiry. This provides an easy authorization handling mechanism for developers. Developers can configure user authorization tokens or service credentials. Service credentials can be set up programmatically or using environment variables. ## Features Token handling in the Industrial IoT SDK provides the following features: - Handling of user tokens - Fetching and handling of technical tokens - Fetching using app specific service credentials - Fetching using tenant specific service credentials - Fetching using tenant specific service credentials with subtenant impersonation - Token validation using issuer, issuing time, expiry time, token algorithm and token type before making API calls - Reuse of technical tokens until they expire and automatic refresh when the expiry time is less than 5 minutes to reduce traffic ### Technical Token Handling Mechanisms #### Token Fetching The Industrial IoT SDK V2 for Java uses the client ID, client secret and other configured parameters when fetching technical tokens to make Industrial IoT API calls. Refer to [Environment Variables required to fetch Technical Tokens](#required-environment-variables-for-fetching-technical-tokens) for more information on parameters to be configured. For more information on Xcelerator domain token handling, refer to the [Xcelerator Adoption in SDK & Platform](xcelerator_sdk_adoption.html) section. The Industrial IoT SDK V2 uses app specific service credentials if available and otherwise looks for tenant specific service credentials. #### Token Validation API calls are only executed by the Industrial IoT SDK if the technical token is valid. The validation uses the issuer, valid issuer, issued at, expiry, token algorithm, and token type in the check. #### Token Caching and Re-Fetching After fetching a valid token, the token is cached in a [`MindsphereCredentials`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/core/MindsphereCredentials.html) object. Every technical token is valid for 30 minutes. A new token is automatically fetched 5 minutes before the expiry. ### Required Environment Variables for Fetching Technical Tokens The Industrial IoT SDK V2 for Java only uses environment variables for fetching tokens if neither user token nor service credentials are available. #### Environment Variables for App Specific Technical Tokens | Environment Variable | Description | | ------------------------------ | --------------------------------------------------------------------------------------- | | `MDSP_KEY_STORE_CLIENT_ID` | Client ID displayed as service credentials in Developer Cockpit or Operator Cockpit | | `MDSP_KEY_STORE_CLIENT_SECRET` | Client secret displayed as service credentials in Developer Cockpit or Operator Cockpit | | `MDSP_OS_VM_APP_NAME` | The application name as stored by the version management of the Operator Services | | `MDSP_OS_VM_APP_VERSION` | The application version as stored by the version management of the Operator Services | | `MDSP_HOST_TENANT` | Host Tenant | | `MDSP_USER_TENANT` | User Tenant | | `MDSP_MANAGEMENT_TENANT` | Management Tenant | | `HOST_BASEDOMAIN`(optional) | The basedomain url for domains other than '\*.mindsphere.io' | #### Environment Variables for Tenant Specific Technical Tokens | Environment Variable | Description | | --------------------------- | ----------------------------------------------------------------- | | `MINDSPHERE_CLIENT_ID` | Client ID of the service credentials | | `MINDSPHERE_CLIENT_SECRET` | Client secret of the service credentials | | `MINDSPHERE_TENANT` | Tenant name | | `HOST_BASEDOMAIN`(optional) | The basedomain url only for domains other than '\*.mindsphere.io' | #### Environment Variables for Tenant Specific Technical Tokens with Subtenant Impersonation | Environmental Variable Name | Description | | --------------------------- | ----------------------------------------------------------------- | | `MINDSPHERE_CLIENT_ID` | Client ID of the service credentials | | `MINDSPHERE_CLIENT_SECRET` | Client secret of the service credentials | | `MINDSPHERE_TENANT` | Tenant name | | `MINDSPHERE_SUB_TENANT` | Subtenant name | | `HOST_BASEDOMAIN`(optional) | The basedomain url only for domains other than '\*.mindsphere.io' | Here are some examples of how you can also pass these variables as parameters for authentication and token handling for Asset Management. Note - `hostBaseDomain` is set to 'mindsphere.io' by default. - `managementTenant` is set to '1000001700' by default ##### For eu1 domain ```java RestClientConfig config = RestClientConfig.builder() .connectionTimeoutInSeconds(100) .hostEnvironment("eu1") .hostBaseDomain("mindsphere.io") .build(); MindsphereCredentials credentials = MindsphereCredentials.appCredentialsBuilder() .appName("myapp") .appVersion("v1.0.0") .keyStoreClientId("tenant1-myapp-v1.0.0") .keyStoreClientSecret("abcdefghijklmnopqrstuvw123") .userTenant("tenant1") .hostTenant("tenant1") .managementTenant("managementTenant001").build(); AssetsClient assets_client = AssetsClient.builder() .mindsphereCredentials(credentials) .restClientConfig(config) .build(); ``` ##### For eu2 domain ```java RestClientConfig config = RestClientConfig.builder() .connectionTimeoutInSeconds(100) .hostEnvironment("eu2") .hostBaseDomain("mindsphere.io") .build(); MindsphereCredentials credentials = MindsphereCredentials.appCredentialsBuilder() .appName("myapp") .appVersion("v1.0.0") .keyStoreClientId("tenant1-myapp-v1.0.0") .keyStoreClientSecret("abcdefghijklmnopqrstuvw123") .userTenant("tenant1") .hostTenant("tenant1") .managementTenant("managementTenant002").build(); AssetsClient assets_client = AssetsClient.builder() .mindsphereCredentials(credentials) .restClientConfig(config) .build(); ``` ##### For private cloud domain For domain "tenant1.abc.basedomain.xyz" ```java RestClientConfig config = RestClientConfig.builder() .connectionTimeoutInSeconds(100) .hostEnvironment("abc") .hostBaseDomain("basedomain.xyz") .build(); MindsphereCredentials credentials = MindsphereCredentials.appCredentialsBuilder() .appName("myapp") .appVersion("v1.0.0") .keyStoreClientId("tenant1-myapp-v1.0.0") .keyStoreClientSecret("abcdefghijklmnopqrstuvw123") .userTenant("tenant1") .hostTenant("tenant1") .managementTenant("managementTenant003").build(); AssetsClient assets_client = AssetsClient.builder() .mindsphereCredentials(credentials) .restClientConfig(config) .build(); ``` # Xcelerator Adoption in SDK & Platform Xcelerator is a comprehensive suite of applications and services designed to transform existing on-premise software products into SaaS-enabled offerings, build new cloud-native XaaS solutions and create a consistent customer experience across Siemens products. ## Adopting Xcelerator Adoption of the Xcelerator platform and services will help us to leverage Xcelerator to enhance and scale applications according to business needs. Major steps in the adoption of Xcelerator include efforts to migrate existing applications to Xcelerator clusters if needed. As all services slowly migrate from the previously used `mindsphere.io` to `sw.siemens.com` domain, applications created on the Onboarding Console will use `sw.siemens.com` domain APIs and resources to use the Xcelerator services capabilities. Customers and API requests to the previous domain following the previous API contracts must be redirected to the new domain instead, which maintains the same APIs, thus without any loss of functionality. Applications previously developed using our SDKs and new applications leveraging the Xcelerator platform capabilities through SDKs can easily adopt the new domain by following the guidelines: ### CASE: Application uses AppCredentials 1. Create a "New Application" in the Onboarding Console. 1. Generate "Application Credentials" with the necessary scopes and permissions to use the corresponding APIs in the Xcelerator domain. 1. Use new credentials in your old application source code. 1. Use with newly released Xcelerator-enabled SDK dependencies. 1. Deploy an application in the new domain. 1. Use the new `basedomain` available in the Xcelerator domain. ### HOST_BASEDOMAIN The `basedomain` URL is set to `siemens.app`. The newly released SDKs, which incorporate Xcelerator domain API requests and Xcelerator-specific token handling, allow you to use the SDK with your Xcelerator domain applications and authentication credentials. ### Key Configuration Notes **Production Pipelines:** ```text Remove MDSP_MANAGEMENT_TENANT variable entirely Use HOST_BASEDOMAIN: siemens.app for Xcelerator platform Store sensitive credentials in GitLab CI/CD variables (masked/protected) ``` **Testing/CI Pipelines:** ```text Include MDSP_MANAGEMENT_TENANT for test environment routing Use HOST_BASEDOMAIN: siemens.app for testing Use dedicated test credentials and tenant IDs ``` The following example demonstrates the use of the new Xcelerator settings: ```java RestClientConfig config = RestClientConfig.builder() .connectionTimeoutInSeconds(100) .socketTimeoutInSeconds(120).headers(map) .hostEnvironment("eu1").hostBaseDomain("siemens.app") .build(); // Production Configuration (Recommended) CoreCredentials credentials = CoreCredentials.appCredentialsBuilder().appName("APPNAME").appVersion("v1.0.0") .keyStoreClientId("TENANT-APPNAME-v1.0.0") .keyStoreClientSecret("xyzxyzxyzxyzxyzxyzxyz") .userTenant("TENANT").hostTenant("TENANT").build(); // Testing/Development Configuration CoreCredentials credentials = CoreCredentials.appCredentialsBuilder().appName("APPNAME").appVersion("v1.0.0") .keyStoreClientId("TENANT-APPNAME-v1.0.0") .keyStoreClientSecret("xyzxyzxyzxyzxyzxyzxyz") .userTenant("TENANT").hostTenant("TENANT") .managementTenant("MANAGEMENT_TENANT").build(); TimeSeriesOperationsClient timeSeriesclient = TimeSeriesOperationsClient.builder() .coreCredentials(credentials) .restClientConfig(config).restTemplate(new RestTemplate()) .build(); ``` # Agent Management Client for Java ## Introduction The Agent Management Java client allows you to onboard, offboard, update and delete agents. It provides connectivity functions to enable communication with Insights Hub. Further implementation of the AgentManagement SDK library has been shown in a sample project that you can download and test in local or on Insights Hub application. Please refer to this repository: [industrial-iot-java-sdk-examples](https://github.com/mindsphere/mindsphere-java-sdk-examples) See [Agent Management](../../../apis/connectivity-agentmanagement/api-agentmanagement-overview.html) for more information about the service. Hint In the IoT context, *assets* are referred to as *entity* and *aspects* as *propertyset*. The following code samples switch between these terms depending on the used APIs.\ Placeholders in the following samples are indicated by angular brackets `< >`. ## Agent Operations The Agents client lists, creates, updates, reads and deletes agents. Client name: [AgentOperationsClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/agentmanagement/apiclient/AgentOperationsClient.html) ### Basic Agent Management #### Create an Agent ```java // Construct the AgentOperationsClient object AgentOperationsClient agents_client = AgentOperationsClient.builder() .mindsphereCredentials() .restClientConfig() .build(); AgentInput agent = null; try { agent = agents_client.agentsPost(); }catch (MindsphereException e) { // Exception handling } ``` #### List all Agents Get all agents for the given filter input. ```java // Construct the AgentOperationsClient object as shown above PagedAgent agents_list = null; try { agents_list = agents_client.agentsGet(, , , ); } catch (MindsphereException e) { // Exception handling } ``` #### Get an Agent by ID ```java // Construct the AgentOperationsClient object as shown above Agent agent = null; try { agent = agents_client.agentsIdGet(); } catch (MindsphereException e) { // Exception handling } ``` #### Update an Agent ```java // Construct the AgentOperationsClient object as shown above Agent agent = null; try { agent = agents_client.agentsIdPut(, , ); } catch (MindsphereException e) { // Exception handling } ``` #### Delete an Agent ```java // Construct the AgentOperationsClient object as shown above try { agents_client.agentsIdDelete(, ); } catch (MindsphereException e) { // Exception handling } ``` #### Get an Agent's Online Status ```java // Construct the AgentOperationsClient object as shown above // Construct the AgentOperationsClient object as shown above OnlineStatus online_status = null; try { online_status = agents_client.agentsIdStatusGet(); } catch (MindsphereException e) { // Exception handling } ``` ## Agent Data Source Configuration Operations The Data Source Configuration client gets, creates and updates the data source configuration. Client name: [DataSourceConfigurationOperationsClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/agentmanagement/apiclient/DataSourceConfigurationOperationsClient.html) ### Get the Data Source Configuration Get the data source configuration for an agent with the given ID. ```java // Construct the DataSourceConfigurationOperationsClient object DataSourceConfigurationOperationsClient data_source_configuration_client = DataSourceConfigurationOperationsClient.builder() .mindsphereCredentials() .restClientConfig() .build(); DataSourceConfiguration data_source_configuration = null; try { data_source_configuration = data_source_configuration_client.agentsIdDataSourceConfigurationGet(); } catch (MindsphereException e) { // Exception handling } ``` ### Create the Data Source Configuration Create the data source configuration for an agent with the given ID. ```java // Construct the DataSourceConfigurationOperationsClient object as shown above //Create a DataSource object DataSource data_sources_item = new DataSource(); data_sources_item.setName(); data_sources_item.setDescription(); data_sources_item.setDataPoints() { data_sources_item.customData() { //Create a DataSourceConfigurationInput object DataSourceConfigurationInput configuration = new DataSourceConfigurationInput(); configuration.setConfigurationId(); configuration.addDataSourcesItem(data_sources_item); DataSourceConfiguration data_source_configuration = null; try { data_source_configuration = data_source_configuration_client.agentsIdDataSourceConfigurationPut(, configuration); } catch (MindsphereException e) { // Exception handling } ``` ### Update the Data Source Configuration Update the data source configuration for an agent with the given ID. ```java // Construct the DataSourceConfigurationOperationsClient object as shown above DataSourceConfiguration data_source_configuration = null; try { data_source_configuration = data_source_configuration_client.agentsIdDataSourceConfigurationPut(, , ); } catch (MindsphereException e) { // Exception handling } ``` ## Agent Boarding Operations The Boarding client is used to offboard an agent and to get the boarding configuration and the boarding status. Client name: [BoardingOperationsClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/agentmanagement/apiclient/BoardingOperationsClient.html) ### Get the Boarding Configuration Get the boarding configuration of an agent with the given ID. ```java // Construct the BoardingOperationsClient object BoardingOperationsClient boarding_client = BoardingOperationsClient.builder() .mindsphereCredentials() .restClientConfig() .build(); ModelConfiguration boarding_configuration = null; try { model_configuration = boarding_client.agentsIdBoardingConfigurationGet(); } catch (MindsphereException e) { // Exception handling } ``` ### Offboard an Agent Offboard an agent with the given ID. ```java // Construct the BoardingOperationsClient object as shown above OnboardingStatus onboarding_status = null; try { onboarding_status = boarding_client.agentsIdBoardingOffboardPost(); } catch (MindsphereException e) { // Exception handling } ``` ### Get the Boarding Status Get the boarding status of an agent with the given ID. ```java // Construct the BoardingOperationsClient object as shown above OnboardingStatus onboarding_status = null; try { onboarding_status = boarding_client.agentsIdBoardingStatusGet(); } catch (MindsphereException e) { // Exception handling } ``` ## Agent Registration Operations The Registration client can register an agent and update agent information. Client name: [RegistrationOperationsClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/agentmanagement/apiclient/RegistrationOperationsClient.html) ### Register an Agent Register an agent. ```java // Construct the RegistrationOperationsClient object RegistrationOperationsClient registration_client = RegistrationOperationsClient.builder() .mindsphereCredentials() .restClientConfig() .build(); ClientIdentifier client_identifier = null; try { client_identifier = registration_client.onBoardAgent(, ); } catch (MindsphereException e) { / ``` Note The `initialAccessToken` string can be obtained via `boardingClient.getBoardingConfiguration(agentId)`. ### Register an Agent with SHARED_SECRET Security Profile ```java // Construct the RegistrationClient object ClientIdentifier client_identifier = null; try { client_identifier = registration_client.onBoardAgentWithSharedSecretProfile(); } catch (MindsphereException e) { / ``` ## Agent Token Operations The Token client is used to get the access token and the key information of OAuth server. Client name: [TokenOperationsClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/agentmanagement/apiclient/TokenOperationsClient.html) ### Fetch an Access Token Fetch an access token. ```java // Construct the TokenOperationsClient object TokenOperationsClient access_token_client = TokenOperationsClient.builder() .mindsphereCredentials() .restClientConfig() .build(); AccessToken access_token = null; try { access_token = access_token_client.oauthTokenPost(, , ); } catch (MindsphereException e) { // Exception handling } ``` ### Provide Key Information of OAuth Server Provide key information of OAuth server. ```java // Construct the TokenOperationsClient object as shown above TokenKey token_key = null; try { token_key = access_token_client.oauthTokenKeyGet(); } catch (MindsphereException e) { // Exception handling } ``` # Asset Management Client for Java ## Introduction The Asset Management Java client allows you to interact with the digital representation of a machine or automation system. The interaction with the Industrial IoT API is abstracted through a Java object model. Refer to [Asset Management](../../../apis/advanced-assetmanagement/api-assetmanagement-overview.html) for more information about the service. Further implementation of the Asset Management SDK library has been shown in a sample project that you can download and test in local or on Insights Hub application. Please refer to this repository: [industrial-iot-java-sdk-examples](https://github.com/mindsphere/mindsphere-java-sdk-examples) Hint Placeholders in the following samples are indicated by angular brackets `< >`. ## Aspect Type Operations The aspect type client manages static and dynamic aspect types. It lists, creates, updates, reads and deletes the aspect types. Client name: [AspecttypeClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/assetmanagement/apiclient/AspecttypeClient.html) ### List all Aspect Types This section shows two ways of getting a list of all aspect types of the tenant. ```java // Construct the AspecttypeClient object AspecttypeClient aspect_type_client = AspecttypeClient.builder() .mindsphereCredentials() .restClientConfig() .build(); AspectTypeListResource aspect_types = null; try { aspect_types = aspect_type_client.listAspectTypes(, , , , ); }catch (MindsphereException e) { // Exception handling } ``` Alternatively, use the [`ListAspectTypesRequest`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/assetmanagement/model/ListAspectTypesRequest.html) model. ```java // Construct the AspecttypeClient object as shown above ListAspectTypesRequest request_object = new ListAspectTypesRequest(); request_object.setPage(); request_object.setSize(); request_object.setFilter(); AspectTypeListResource aspectTypes = null; try { aspect_types = aspect_type_client.listAspectTypes(request_object); }catch (MindsphereException e) { // Exception handling } ``` ### Create an Aspect Type This section shows two ways of creating an aspect type. ```java // Construct the AspecttypeClient object as shown above AspectTypeResource aspect_type_resource = null; try { aspect_type_resource = aspect_type_client.saveAspectType(, , ); } catch (MindsphereException e) { // Exception handling } ``` Alternatively, use the [`SaveAspectTypeRequest`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/assetmanagement/model/SaveAspectTypeRequest.html) model. ```java // Construct the AspecttypeClient object as shown above SaveAspectTypeRequest request_object = new SaveAspectTypeRequest(); request_object.setId(); request_object.setAspecttype(); AspectTypeResource aspect_type_resource = null; try { aspect_type_resource = aspect_type_client.saveAspectType(request_object); }catch (MindsphereException e) { // Exception handling } ``` ### Update an Aspect Type This section shows two options for updating an existing aspect type. Variables can only be added, but not removed or renamed. ```java // Construct the AspecttypeClient object as shown above AspectTypeResource aspect_type_resource = null; try { aspect_type_resource = aspect_type_client.updateAspectType(, , ); } catch (MindsphereException e) { // Exception handling } ``` Alternatively, use the [`UpdateAspectTypeRequest`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/assetmanagement/model/UpdateAspectTypeRequest.html) model. ```java // Construct the AspecttypeClient object as shown above UpdateAspectTypeRequest request_object = new UpdateAspectTypeRequest(); request_object.setId(); request_object.setAspecttype(); request_object.setIfMatch(); AspectTypeResource aspect_type_resource = null; try { aspect_type_resource = aspect_type_client.updateAspectType(request_object); }catch (MindsphereException e) { // Exception handling } ``` ### Read an Aspect Type This section shows two options for reading an aspect type. ```java // Construct the AspecttypeClient object as shown above AspectTypeResource aspect_type_resource = null; try { aspect_type_resource = aspect_type_client.getAspectType(, ); } catch (MindsphereException e) { // Exception handling } ``` Alternatively, use the [`GetAspectTypeRequest`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/assetmanagement/model/GetAspectTypeRequest.html) model. ```java // Construct the AspecttypeClient object as shown above GetAspectTypeRequest request_object = new GetAspectTypeRequest(); request_object.setId(); AspectTypeResource aspect_type_resource = null; try { aspect_type_resource = aspect_type_client.getAspectType(request_object); }catch (MindsphereException e) { // Exception handling } ``` ### Delete an Aspect Type This section shows two options for deleting an aspect type. An aspect type can only be deleted, if no asset type uses it. ```java // Construct the AspecttypeClient object as shown above try { aspect_type_client.deleteAspectType(, ); } catch (MindsphereException e) { // Exception handling } ``` Alternatively, use the [`DeleteAspectTypeRequest`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/assetmanagement/model/DeleteAspectTypeRequest.html) model. ```java // Construct the AspecttypeClient object as shown above DeleteAspectTypeRequest request_object = new DeleteAspectTypeRequest(); request_object.setId(); request_object.setIfMatch(); try { aspect_type_client.deleteAspectType(request_object); }catch (MindsphereException e) { // Exception handling } ``` ### Get Aspect Types using Filters #### Get Aspect Types with a Specific Parameter This code sample uses the `filter - equals` functionality to get all aspect types whose name or tenant ID is equal to the input value. ```java // Construct the AspecttypeClient object as shown above AspectTypeListResource aspect_types = null; try { aspect_types = aspect_type_client.getAspectTypesEqualTo(FieldTypeEnum.NAME, ); }catch (MindsphereException e) { // Exception handling } ``` #### Get Aspect Types with a Parameter from a List Use the `filter - in` functionality to get all aspect types whose name or tenant ID matches any value in an array of values. ```java // Construct the AspecttypeClient object as shown above AspectTypeListResource aspect_types = null; try { aspect_types = aspect_type_client.getAspectTypesLike(FieldTypeEnum.NAME, , ); }catch (MindsphereException e) { // Exception handling } ``` #### Get Aspect Types where a Parameter Starts with a Specific Value Use the `filter - startsWith` functionality to get all aspect types whose name or tenant ID starts with an input value. ```java // Construct the AspecttypeClient object as shown above AspectTypeListResource aspect_types = null; try { aspect_types = aspect_type_client.getAspectTypesStartsWith(FieldTypeEnum.NAME, ); }catch (MindsphereException e) { // Exception handling } ``` #### Get Aspect Types where a Parameter Ends with a Specific Value Use the `filter - endsWith` functionality to get all aspect types whose name or tenant ID ends with an input value. ```java // Construct the AspecttypeClient object as shown above AspectTypeListResource aspect_types = null; try { aspect_types = aspect_type_client.getAspectTypesEndsWith(FieldTypeEnum.NAME, ); }catch (MindsphereException e) { // Exception handling } ``` #### Get Aspect Types where a Parameter Contains a Specific Value Use the `filter - contains` functionality of Asset management to get all aspect types whose name or tenantId contains an input value. ```java // Construct the AspecttypeClient object as shown above AspectTypeListResource aspect_types = null; try { aspect_types = aspect_type_client.getAspectTypesContains(FieldTypeEnum.NAME, ); }catch (MindsphereException e) { // Exception handling } ``` ## Asset Type Operations The asset type client manages asset types. It lists, creates, updates, reads and deletes the asset types. It can also be used to add and delete file assignments to the asset types. Client name: [AssettypeClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/assetmanagement/apiclient/AssettypeClient.html) ### List all Asset Types This section shows two options for listing all asset types of the tenant. ```java // Construct the AssettypeClient object AssettypeClient asset_type_client = AssettypeClient.builder() .mindsphereCredentials() .restClientConfig() .build(); AssetTypeListResource asset_types = null; try { asset_types = asset_type_client.listAssetTypes(, , , , , ); } catch (MindsphereException e) { // Exception handling } ``` Alternatively, use the [`ListAssetTypesRequest`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/assetmanagement/model/ListAssetTypesRequest.html) model. ```java // Construct the AssettypeClient object as shown above ListAssetTypesRequest request_object = new ListAssetTypesRequest(); request_object.setPage(); request_object.setSize(); AssetTypeListResource asset_types = null; try { asset_types = asset_type_client.listAssetTypes(request_object); } catch (MindsphereException e) { // Exception handling } ``` ### Create an Asset Type This section shows two options for creating an asset type. ```java // Construct the AssettypeClient object as shown above AssetTypeResource asset_type_resource = null try { asset_type_resource = asset_type_client.saveAssetType(, , ); } catch (MindsphereException e) { // Exception handling } ``` Alternatively, use the [`SaveAssetTypeRequest`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/assetmanagement/model/SaveAssetTypeRequest.html) model. ```java // Construct the AssettypeClient object as shown above SaveAssetTypeRequest request_object = new SaveAssetTypeRequest(); request_object.setId(); request_object.setAssettype(); AssetTypeResource asset_type_resource = null try { asset_type_resource = asset_type_client.saveAssetType(request_object); } catch (MindsphereException e) { // Exception handling } ``` ### Update an Asset Type This section shows two options for updating an existing asset type. ```java // Construct the AssettypeClient object as shown above AssetTypeResource asset_type_resource = null; try { asset_type_resource = asset_type_client.updateAssetType(, , ); } catch (MindsphereException e) { // Exception handling } ``` Alternatively, use the [`UpdateAssetTypeRequest`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/assetmanagement/model/UpdateAssetTypeRequest.html) model. ```java // Construct the AssettypeClient object as shown above UpdateAssetTypeRequest request_object = new UpdateAssetTypeRequest(); request_object.setId(); request_object.setAssettype(); request_object.setIfMatch(); AssetTypeResource asset_type_resource = null; try { asset_type_resource = asset_type_client.updateAssetType(request_object); } catch (MindsphereException e) { // Exception handling } ``` ### Read an Asset Type This section shows two options for reading an asset type. ```java // Construct the AssettypeClient object as shown above AssetTypeResource asset_type_resource = null; try { asset_type_resource = asset_type_client.getAssetType(, , ); } catch (MindsphereException e) { // Exception handling } ``` Alternatively, use the [`GetAssetTypeRequest`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/assetmanagement/model/GetAssetTypeRequest.html) model. ```java // Construct the AssettypeClient object as shown above GetAssetTypeRequest request_object = new GetAssetTypeRequest(); request_object.setId(); AssetTypeResource asset_type_resource = null; try { asset_type_resource = asset_type_client.getAssetType(request_object); } catch (MindsphereException e) { // Exception handling } ``` ### Delete an Asset Type This section shows two options for deleting an asset type. Deletion is only possible for asset types without children and when there is no asset instantiating them. ```java // Construct the AssettypeClient object as shown above try { asset_type_client.deleteAssetType(, ); } catch (MindsphereException e) { // Exception handling } ``` Alternatively, use the [`DeleteAssetTypeRequest`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/assetmanagement/model/DeleteAssetTypeRequest.html) model. ```java // Construct the AssettypeClient object as shown above DeleteAssetTypeRequest request_object = new DeleteAssetTypeRequest(); request_object.setId(); request_object.setIfMatch(); try { asset_type_client.deleteAssetType(request_object); } catch (MindsphereException e) { // Exception handling } ``` ### Add a New File Assignment to an Asset Type This section shows two options for adding a new file assignment to an asset type. By default, this file is assigned to asset types extending this type and assets instantiated from it. ```java // Construct the AssettypeClient object as shown above AssetTypeResource asset_type_resource = null; try { asset_type_resource = asset_type_client.saveAssetTypeFileAssignment(, , , ); } catch (MindsphereException e) { // Exception handling } ``` Alternatively, use the [`SaveAssetTypeFileAssignmentRequest`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/assetmanagement/model/SaveAssetTypeFileAssignmentRequest.html) model. ```java // Construct the AssettypeClient object as shown above SaveAssetTypeFileAssignmentRequest request_object = new SaveAssetTypeFileAssignmentRequest(); request_object.setId(); request_object.setKey(); request_object.setAssignment(); request_object.setIfMatch(); AssetTypeResource asset_type_resource = null; try { asset_type_resource = asset_type_client.saveAssetTypeFileAssignment(request_object); } catch (MindsphereException e) { // Exception handling } ``` ### Delete a File Assignment from an Asset Type This section shows two options for deleting a file assignment from an asset type. If the file is assigned to the asset type's parent, its key will be displayed in the inherited value field. ```java // Construct the AssettypeClient object as shown above AssetTypeResource asset_type_resource = null; try { asset_type_resource = asset_type_client.deleteAssetTypeFileAssignment(, , ); } catch (MindsphereException e) { // Exception handling } ``` Alternatively, use the [`DeleteAssetTypeFileAssignmentRequest`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/assetmanagement/model/DeleteAssetTypeFileAssignmentRequest.html) model. ```java // Construct the AssettypeClient object as shown above DeleteAssetTypeFileAssignmentRequest request_object = new DeleteAssetTypeFileAssignmentRequest(); request_object.setId(); request_object.setKey(); request_object.setIfMatch(); AssetTypeResource asset_type_resource = null; try { asset_type_resource = asset_type_client.deleteAssetTypeFileAssignment(request_object); } catch (MindsphereException e) { // Exception handling } ``` ### Get Asset Types using Filters #### Get Asset Types with a Specific Parameter Use the `filter - equals` functionality to get all asset types whose name or tenant ID or parent type ID is equal to the input value. ```java // Construct the AssettypeClient object as shown above AssetTypeListResource asset_types = null; try { asset_types = asset_type_client.getAssetTypesEqualTo(FieldTypeEnum.NAME, ); }catch (MindsphereException e) { // Exception handling } ``` #### Get Asset Types with a Parameter from a List Use the `filter - in` functionality to get all asset types whose name or tenant ID or parent type ID matches any value in an array of values. ```java // Construct the AssettypeClient object as shown above AssetTypeListResource asset_types = null; try { asset_types = asset_type_client.getAssetTypesLike(FieldTypeEnum.NAME, , ); }catch (MindsphereException e) { // Exception handling } ``` #### Get Asset Types where a Parameter Starts with a Specific Value Use the `filter - startsWith` functionality to get all asset types whose name or tenant ID or parent type ID starts with an input value. ```java // Construct the AssettypeClient object as shown above AssetTypeListResource asset_types = null; try { asset_types = asset_type_client.getAssetTypesStartsWith(FieldTypeEnum.NAME, ); }catch (MindsphereException e) { // Exception handling } ``` #### Get Asset Types where a Parameter Ends with a Specific Value Use the `filter - endsWith` functionality to get all asset types whose name or tenant ID or parent type ID ends with an input value. ```java // Construct the AssettypeClient object as shown above AssetTypeListResource asset_types = null; try { asset_types = asset_type_client.getAssetTypesEndsWith(FieldTypeEnum.NAME, ); }catch (MindsphereException e) { // Exception handling } ``` #### Get Asset Types where a Parameter Contains a Specific Value Use the `filter - contains` functionality to get all asset types whose name or tenant ID or parent type ID contains an input value. ```java // Construct the AssettypeClient object as shown above AssetTypeListResource asset_types = null; try { asset_types = asset_type_client.getAssetTypesContains(FieldTypeEnum.NAME, ); }catch (MindsphereException e) { // Exception handling } ``` ## Asset Operations The asset client manages the user's assets and their locations. Three different types of assets can be created: device types, agent types and hierarchy types. The asset client lists, creates, updates, reads, deletes, moves and replaces the asset. It can also be used to add and delete file assignments to assets. Client name: [AssetsClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/assetmanagement/apiclient/AssetsClient.html) ### List all Assets This section shows two options for listing all assets available for the authenticated user. ```java // Construct the AssetClient object AssetsClient asset_client = AssetsClient.builder() .mindsphereCredentials() .restClientConfig() .build(); AssetListResource assets = null; try{ assets = asset_client.listAssets(, , , , ); } catch (MindsphereException e) { // Exception handling } ``` Alternatively, use the [`ListAssetsRequest`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/assetmanagement/model/ListAssetsRequest.html) model. ```java // Construct the AssetClient object as shown above ListAssetsRequest request_object = new ListAssetsRequest(); request_object.setPage(); request_object.setSize(); AssetListResource assets = null; try{ assets = asset_client.listAssets(request_object); } catch (MindsphereException e) { // Exception handling } ``` ### Create an Asset This section shows two options for creating a new asset with the provided content. ```java // Construct the AssetClient object as shown above AssetResourceWithHierarchyPath asset_resource = null; try{ asset_resource = asset_client.addAsset(); } catch (MindsphereException e) { // Exception handling } ``` Alternatively, use the [`AddAssetRequest`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/assetmanagement/model/AddAssetRequest.html) model. ```java // Construct the AssetClient object as shown above AddAssetRequest request_object = new AddAssetRequest(); request_object.setAsset(); AssetResourceWithHierarchyPath asset_resource = null; try{ asset_resource = asset_client.addAsset(request_object); } catch (MindsphereException e) { // Exception handling } ``` ### Read an Asset This section shows two options for reading a single asset. All static properties of the asset are returned. ```java // Construct the AssetClient object as shown above AssetResourceWithHierarchyPath asset_resource = null; try{ asset_resource = asset_client.getAsset(, ); } catch (MindsphereException e) { // Exception handling } ``` Alternatively, use the [`GetAssetRequest`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/assetmanagement/model/GetAssetRequest.html) model. ```java // Construct the AssetClient object as shown above GetAssetRequest request_object = new GetAssetRequest(); request_object.setId(); AssetResourceWithHierarchyPath asset_resource = null; try{ asset_resource = asset_client.getAsset(request_object); } catch (MindsphereException e) { // Exception handling } ``` ### Update an Asset This section shows two options for updating an existing asset with the provided content. Only values can be modified, but not the asset structure. The asset structure can be modified in the asset type. ```java // Construct the AssetClient object as shown above AssetResourceWithHierarchyPath asset_resource = null; try{ asset_resource = asset_client.updateAsset(, , ); } catch (MindsphereException e) { // Exception handling } ``` Alternatively, use the [`UpdateAssetRequest`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/assetmanagement/model/UpdateAssetRequest.html) model. ```java // Construct the AssetClient object as shown above UpdateAssetRequest request_object = new UpdateAssetRequest(); request_object.setId(); request_object.setAsset(); request_object.setIfMatch(); AssetResourceWithHierarchyPath asset_resource = null; try{ asset_resource = asset_client.updateAsset(request_object); } catch (MindsphereException e) { // Exception handling } ``` ### Delete an Asset This section shows two options for deleting an existing asset. After deletion, users with an admin role can still read the asset, but modification is not possible anymore. It is not possible to delete an asset if it has children. ```java // Construct the AssetClient object as shown above try{ asset_client.deleteAsset(, ); } catch (MindsphereException e) { // Exception handling } ``` Alternatively, use the [`DeleteAssetRequest`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/assetmanagement/model/DeleteAssetRequest.html) model. ```java // Construct the AssetClient object as shown above DeleteAssetRequest request_object = new DeleteAssetRequest(); request_object.setId(); request_object.setIfMatch(); try{ asset_client.deleteAsset(request_object); } catch (MindsphereException e) { // Exception handling } ``` ### Move an Asset This section shows two options for moving an existing asset and all of its children in the instance hierarchy. ```java // Construct the AssetClient object as shown above AssetResourceWithHierarchyPath asset_resource = null; try{ asset_resource = asset_client.moveAsset(, , ); } catch (MindsphereException e) { // Exception handling } ``` Alternatively, use the [`MoveAssetRequest`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/assetmanagement/model/MoveAssetRequest.html) model. ```java // Construct the AssetClient object as shown above MoveAssetRequest request_object = new MoveAssetRequest(); request_object.setId(); request_object.setMoveParameters(); request_object.setIfMatch(); AssetResourceWithHierarchyPath asset_resource = null; try{ asset_resource = asset_client.moveAsset(request_object); } catch (MindsphereException e) { // Exception handling } ``` ### Replace an Asset This section shows two options for updating an asset with the provided content. ```java // Construct the AssetClient object as shown above AssetResourceWithHierarchyPath asset_resource = null; try{ asset_resource = asset_client.replaceAsset(, , ); } catch (MindsphereException e) { // Exception handling } ``` Alternatively, use the [`ReplaceAssetRequest`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/assetmanagement/model/ReplaceAssetRequest.html) model. ```java // Construct the AssetClient object as shown above ReplaceAssetRequest request_object = new ReplaceAssetRequest(); request_object.setId(); request_object.setAsset(); request_object.setIfMatch(); AssetResourceWithHierarchyPath asset_resource = null; try{ asset_resource = asset_client.replaceAsset(request_object); } catch (MindsphereException e) { // Exception handling } ``` ### Get the User's Root Asset This section shows two options for reading the user's root asset, from which the whole asset hierarchy can be rebuilt. ```java // Construct the AssetClient object as shown above RootAssetResource root_asset_resource = null; try{ root_asset_resource = asset_client.getRootAsset(); } catch (MindsphereException e) { // Exception handling } ``` Alternatively, use the [`GetRootAssetRequest`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/assetmanagement/model/GetRootAssetRequest.html) model. ```java // Construct the AssetClient object as shown above GetRootAssetRequest request_object = new GetRootAssetRequest(); request_object.setIfNoneMatch(); RootAssetResource root_asset_resource = null; try{ root_asset_resource = asset_client.getRootAsset(request_object); } catch (MindsphereException e) { // Exception handling } ``` ### Add a New File Assignment to an Asset This section shows two options for adding a new file assignment to an asset. The file is also assigned to all children of the asset by default. ```java // Construct the AssetClient object as shown above AssetResourceWithHierarchyPath asset_resource = null; try { asset_resource = asset_client.saveAssetFileAssignment(, , , ); } catch (MindsphereException e) { // Exception handling } ``` Alternatively, use the [`SaveAssetFileAssignmentRequest`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/assetmanagement/model/SaveAssetFileAssignmentRequest.html) model. ```java // Construct the AssetClient object as shown above SaveAssetFileAssignmentRequest request_object = new SaveAssetFileAssignmentRequest(); request_object.setId(); request_object.setKey(); request_object.setAssignment(); request_object.setIfMatch(h); AssetResourceWithHierarchyPath asset_resource = null; try { asset_resource = asset_client.saveAssetFileAssignment(request_object); } catch (MindsphereException e) { // Exception handling } ``` ### Delete a File Assignment from an Asset This section shows two options for deleting a file assignment from an asset. ```java // Construct the AssetClient object as shown above AssetResourceWithHierarchyPath asset_resource = null; try { asset_resource = asset_client.deleteAssetFileAssigment(, , ); } catch (MindsphereException e) { // Exception handling } ``` Alternatively, use the [`DeleteAssetFileAssigmentRequest`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/assetmanagement/model/DeleteAssetFileAssigmentRequest.html) model. ```java // Construct the AssetClient object as shown above DeleteAssetFileAssigmentRequest request_object = new DeleteAssetFileAssigmentRequest(); request_object.setId(response.getAssetId()); request_object.setKey(); request_object.setIfMatch(); AssetResourceWithHierarchyPath asset_resource = null; try { asset_resource = asset_client.deleteAssetFileAssigment(request_object); } catch (MindsphereException e) { // Exception handling } ``` ### Get Assets using Filters #### Get Assets with a Specific Parameter Use the `filter - equals` functionality to get all assets whose properties like name, tenantId, assetId, externalId, subtenant is equal to the input value. ```java // Construct the AssetClient object as shown above AssetListResource assets = null; try { assets = asset_client.getAssetsEqualTo(FieldTypeEnum.NAME, ); }catch (MindsphereException e) { // Exception handling } ``` #### Get Assets with a Parameter from a List Use the `filter - in` functionality to get all assets whose properties like name, tenantId, assetId, externalId, subtenant matches any value in an array of values. ```java // Construct the AssetClient object as shown above AssetListResource assets = null; try { assets = asset_client.getAssetsLike(FieldTypeEnum.NAME, , ); }catch (MindsphereException e) { // Exception handling } ``` #### Get Assets where a Parameter Starts with a Specific Value Use the `filter - startsWith` functionality to get all assets whose name or tenantId or parentTypeId starts with an input value. ```java // Construct the AssetClient object as shown above AssetListResource assets = null; try { assets = asset_client.getAssetsStartsWith(FieldTypeEnum.NAME, ); }catch (MindsphereException e) { // Exception handling } ``` #### Get Assets where a Parameter Ends with a Specific Value Use the `filter - endsWith` functionality to get all assets whose name or tenant ID or parent type ID ends with an input value. ```java // Construct the AssetClient object as shown above AssetListResource assets = null; try { assets = asset_client.getAssetsEndsWith(FieldTypeEnum.NAME, ); }catch (MindsphereException e) { // Exception handling } ``` #### Get Assets where a Parameter Contains a Specific Value Use the `filter - contains` functionality to get all assets whose name or tenant ID or parent type ID contains an input value. ```java // Construct the AssetClient object as shown above AssetListResource assets = null; try { assets = asset_client.getAssetsContains(FieldTypeEnum.NAME, ); }catch (MindsphereException e) { // Exception handling } ``` #### Get Assets of a Specific Type Use the `filter - hasType` functionality to get all assets of a particular type. ```java // Construct the AssetClient object as shown above AssetListResource assets = null; try { assets = asset_client.getAssetsOfType(); }catch (MindsphereException e) { // Exception handling } ``` ## Structure Operations The structure client is used to list an asset's aspects and variables. Client name: [StructureClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/assetmanagement/apiclient/StructureClient.html) ### Get an Asset's Variables This section shows two options for getting all variables of an asset. ```java // Construct the StructureClient object StructureClient structure_client = StructureClient.builder() .mindsphereCredentials() .restClientConfig() .build(); VariableListResource structure_variables = null; try{ structure_variables = structure_client.listAssetVariables(, , , , , ); } catch (MindsphereException e) { // Exception handling } ``` Alternatively, use the [`ListAssetVariablesRequest`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/assetmanagement/model/ListAssetVariablesRequest.html) model. ```java // Construct the StructureClient object as shown above ListAssetVariablesRequest request_object = new ListAssetVariablesRequest(); request_object.setId(assetId()); VariableListResource structure_variables = null; try{ structure_variables = structure_client.listAssetVariables(request_object); } catch (MindsphereException e) { // Exception handling } ``` ### Get all Aspects of an Asset This section shows two options for getting all static and dynamic aspects of an asset. ```java // Construct the StructureClient object as shown above AspectListResource structure_aspects = null; try{ structure_aspects = structure_client.listAssetAspects(, , , , , ); } catch (MindsphereException e) { // Exception handling } ``` Alternatively, use the [`ListAssetAspectsRequest`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/assetmanagement/model/ListAssetAspectsRequest.html) model. ```java // Construct the StructureClient object as shown above ListAssetAspectsRequest request_object = new ListAssetAspectsRequest(); request_object.setId(assetId()); AspectListResource structure_aspects = null; try{ structure_aspects = structure_client.listAssetAspects(request_object); } catch (MindsphereException e) { // Exception handling } ``` ## Locations Operations The locations client manages the location of an asset. It can be used to create, update and delete an asset's location. Client name: [LocationsClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/assetmanagement/apiclient/LocationsClient.html) ### Create or Update an Asset's Location This section shows two options for creating or updating the location that is assigned to an asset. If no location is defined for the asset yet, a new one is created. Otherwise, it is updated. ```java // Construct the AssetLocationClient object LocationsClient locations_client = LocationsClient.builder() .mindsphereCredentials() .restClientConfig() .build(); try{ locations_client.saveAssetLocation(, , ); } catch (MindsphereException e) { // Exception handling } ``` Alternatively, use the [`SaveAssetLocationRequest`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/assetmanagement/model/SaveAssetLocationRequest.html) model. ```java // Construct the AssetLocationClient object as shown above SaveAssetLocationRequest request_object = new SaveAssetLocationRequest(); request_object.setId(); request_object.setIfMatch(); request_object.setLocation(); try{ locations_client.saveAssetLocation(request_object); } catch (MindsphereException e) { // Exception handling } ``` ### Delete an Asset's Location This section shows two options for deleting the location from an asset. ```java // Construct the AssetLocationClient object as shown above AssetResourceWithHierarchyPath asset_resource = null; try{ asset_resource = locations_client.deleteAssetLocation(, ); } catch (MindsphereException e) { // Exception handling } ``` Alternatively, use the [`DeleteAssetLocationRequest`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/assetmanagement/model/DeleteAssetLocationRequest.html) model. ```java // Construct the AssetLocationClient object as shown above DeleteAssetLocationRequest request_object = new DeleteAssetLocationRequest(); request_object.setId(); request_object.setIfMatch(); AssetResourceWithHierarchyPath asset_resource = null; try{ asset_resource = locations_client.deleteAssetLocation(request_object); } catch (MindsphereException e) { // Exception handling } ``` ## Billboard Operations The billboard client can be used to list all available resources. Client name: [BillboardClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/assetmanagement/apiclient/BillboardClient.html) ### List all Available Resources List all links for available resources ```java // Construct the BillboardClient object BillboardClient billboard_client = BillboardClient.builder() .mindsphereCredentials() .restClientConfig() .build(); BillboardResource billboard_resource = null; try{ billboard_resource = billboard_client.getBillboard(); } catch (MindsphereException e) { // Exception handling } ``` ## File Operations The files client manages files that can be assigned to a resource. It is used to list, read, upload, delete, replace and download a file. Client name: [FilesClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/assetmanagement/apiclient/FilesClient.html) ### List all Files This section shows two options for getting metadata of all uploaded files. ```java // Construct the FilesClient object FilesClient files_client = FilesClient.builder() .mindsphereCredentials() .restClientConfig() .build(); FileMetadataListResource file_metadata_list_resource = null; try{ file_metadata_list_resource = files_client.listFiles(, , , , ); } catch (MindsphereException e) { // Exception handling } ``` Alternatively, use the [`ListFilesRequest`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/assetmanagement/model/ListFilesRequest.html) model. ```java // Construct the FilesClient object as shown above ListFilesRequest request_object = new ListFilesRequest(); request_object.setPage(); request_object.setSize(); FileMetadataListResource file_metadata_list_resource = null; try{ file_metadata_list_resource = files_client.listFiles(request_object); } catch (MindsphereException e) { // Exception handling } ``` ### Get a File This section shows two options for getting the metadata of a file with the user defined ID. ```java // Construct the FilesClient object as shown above FileMetadataResource file_metadata_resource = null; try{ file_metadata_resource = files_client.getFile(, ); } catch (MindsphereException e) { // Exception handling } ``` Alternatively, use the [`GetFileRequest`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/assetmanagement/model/GetFileRequest.html) model. ```java // Construct the FilesClient object as shown above GetFileRequest request_object = new GetFileRequest(); request_object.setFileId(); FileMetadataResource file_metadata_resource = null; try{ file_metadata_resource = files_client.getFile(request_object); } catch (MindsphereException e) { // Exception handling } ``` ### Download a File This section shows two options for returning a file by its ID. ```java // Construct the FilesClient object as shown above byte[] file_contents = null; try{ file_contents = files_client.downloadFile(); } catch (MindsphereException e) { // Exception handling } ``` Alternatively, use the [`DownloadFileRequest`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/assetmanagement/model/DownloadFileRequest.html) model. ```java // Construct the FilesClient object as shown above DownloadFileRequest request_object = new DownloadFileRequest(); request_object.setFileId(); byte[] file_contents = null; try{ file_contents = files_client.downloadFile(request_object); } catch (MindsphereException e) { // Exception handling } ``` ### Replace a File This section shows two options for updating a previously uploaded file. The maximum file size is 5 MB. ```java // Construct the FilesClient object as shown above FileMetadataResource file_metadata_resource = null; try{ file_metadata_resource = files_client.replaceFile(, , , , , ); } catch (MindsphereException e) { // Exception handling } ``` Alternatively, use the [`ReplaceFileRequest`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/assetmanagement/model/ReplaceFileRequest.html) model. ```java // Construct the FilesClient object as shown above ReplaceFileRequest request_object = new ReplaceFileRequest(); request_object.setIfMatch(); request_object.setFileId(); request_object.setFile(); request_object.setName(); request_object.setScope(); FileMetadataResource file_metadata_resource = null; try{ file_metadata_resource = files_client.replaceFile(request_object); } catch (MindsphereException e) { // Exception handling } ``` ### Upload a File This section shows two options for uploading a file to be used in Asset Management. ```java // Construct the FilesClient object as shown above FileMetadataResource file_metadata_resource = null; try{ file_metadata_resource = files_client.uploadFile(, , , ); } catch (MindsphereException e) { // Exception handling } ``` Alternatively, use the [`UploadFileRequest`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/assetmanagement/model/UploadFileRequest.html) model. ```java // Construct the FilesClient object as shown above UploadFileRequest request_object = new UploadFileRequest(); request_object.setFile(); request_object.setName(); FileMetadataResource file_metadata_resource = null; try{ file_metadata_resource = files_client.uploadFile(request_object); } catch (MindsphereException e) { // Exception handling } ``` ### Delete a File This section shows two options for deleting a file. ```java // Construct the FilesClient object as shown above try{ deleteFile(, ); } catch (MindsphereException e) { // Exception handling } ``` Alternatively, use the [`DeleteFileRequest`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/assetmanagement/model/DeleteFileRequest.html) model. ```java // Construct the FilesClient object as shown above DeleteFileRequest request_object = new DeleteFileRequest(); request_object.setIfMatch(); request_object.setFileId(); try{ deleteFile(request_object); } catch (MindsphereException e) { // Exception handling } ``` ## Page Iterators The page iterators for the Asset Management API client can be used to get a specific page, the next page or the previous page of data. The page iterators support the following operations: - `getPage(Integer pageNumber);` - `next();` - `previous();` The following iterators are provided for different controllers in Asset Management: - [AspectTypePageIterator](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/assetmanagement/iterator/model/AspectTypePageIterator.html) - [AssetTypePageIterator](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/assetmanagement/iterator/model/AssetTypePageIterator.html) - [AssetPageIterator](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/assetmanagement/iterator/model/AssetPageIterator.html) - [StructureAspectsPageIterator](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/assetmanagement/iterator/model/StructureAspectsPageIterator.html) - [StructureVariablesPageIterator](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/assetmanagement/iterator/model/StructureVariablesPageIterator.html) ```java try { // Construct the AssetsClient object AssetsClient asset_client = AssetsClient.builder() .mindsphereCredentials() .restClientConfig() .build(); // Create Asset Page Iterator AssetPageIterator asset_page_iterator = new AssetPageIterator(asset_client, ); AssetListResource assets = asset_page_iterator.getPage(10); } catch (MindsphereException e) { // Exception handling } ``` # Insights Hub Credentials ## Introduction The Insights Hub Credentials contain details for fetching new tokens and store fetched tokens. Hint Placeholders in the following samples are indicated by angular brackets `< >`. ## Creating MindsphereCredentials Objects Object Name: [MindsphereCredentials](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/core/MindsphereCredentials.html) ### Creating MindsphereCredentials using a User Token ```java MindsphereCredentials credentials = MindsphereCredentials.userTokenBuilder() .authorization() .build(); ``` Deprecated Method The following method for creating service credentials using a user token is deprecated and shall not be used anymore: ```java MindsphereCredentials credentials = MindsphereCredentials.builder() .authorization() .build(); ``` ### Creating MindsphereCredentials using App Specific Service Credentials The app specific service credentials are obtained from [Developer Cockpit](https://documentation.mindsphere.io/MindSphere/system-tools/developer-cockpit.html) or [Operator Cockpit](https://documentation.mindsphere.io/resources/html/operator-cockpit/en-US/index.html). During the development of an application you may provide the service credentials programmatically. The final application should obtain the client ID, client secret, app name and app version from the [environment variables](../token_handling_v2.html#environment-variables-for-app-specific-technical-tokens) and the host tenant and user tenant from the user token as shown below. ```java MindsphereCredentials credentials = MindsphereCredentials.appCredentialsBuilder() .keyStoreClientId() .keyStoreClientSecret() .appName() .appVersion() .hostTenant() .userTenant() .build(); ``` ```java MindsphereCredentials credentials = MindsphereCredentials.appCredentialsBuilder() .authorization() .build(); ``` Note The host tenant and user tenant values should not be hard-coded in the application as they are subject to change. `MindSphereCredentials` objects can extract these values from the user token provided as `authorization` parameter. ### Creating MindsphereCredentials using Tenant Specific Service Credentials The tenant specific service credentials must be provided programmatically or via [environment variables](../token_handling_v2.html#environment-variables-for-tenant-specific-technical-tokens). Tenant specific service credentials are obtained as described in [Accessing Industrial IoT APIs with Service Credentials](https://developer.mindsphere.io/howto/howto-selfhosted-api-access.html#creating-service-credentials). If credentials are set via environment variables, the [`MindsphereCredentials`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/core/MindsphereCredentials.html) object does not need to be build. ```java MindsphereCredentials credentials = MindsphereCredentials.tenantCredentialsBuilder() .clientId() .clientSecret() .tenant() .build(); ``` Deprecated Method The following method for creating tenant specific service credentials is deprecated and shall not be used anymore: ```java MindsphereCredentials credentials = MindsphereCredentials.builder() .clientId() .clientSecret() .tenant() .build(); ``` ### Creating MindsphereCredentials using Tenant specific Service Credentials for Subtenant Impersonation The tenant specific service credentials for subtenant impersonation must be provided programmatically or via [environment variables](../token_handling_v2.html#environment-variables-for-tenant-specific-technical-tokens-with-subtenant-impersonation). ```java MindsphereCredentials credentials = MindsphereCredentials.tenantCredentialsBuilder() .clientId() .clientSecret() .tenant() .subTenant() .tokenType(TokenScope.SUB_TENANT) .build(); ``` # Event Management Client for Java ## Introduction The Event Management Java client allows you to manage events associated with assets and event types. Refer to [Event Management](../../../apis/advanced-eventmanagement/api-eventmanagement-overview.html) for more information about the service. Further implementation of the Event Management SDK library has been shown in a sample project that you can download and test in local or on Insights Hub application. Please refer to this repository: [industrial-iot-java-sdk-examples](https://github.com/mindsphere/mindsphere-java-sdk-examples) Hint Placeholders in the following samples are indicated by angular brackets `< >`. ## Event Type Operations The Event Types client creates, reads, updates and deletes event types. Client name: [EventTypesClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/eventmanagement/apiclient/EventTypesClient.html) ### List all Event Types ```java // Construct EventTypesClient object EventTypesClient eventTypesClient = EventTypesClient.builder() .mindsphereCredentials() .restClientConfig() .build(); EventType event_types = null; try { event_types = eventTypeClient.getEventTypes(, , "", "", ""); } catch (MindsphereException e) { // Exception Handling } ``` ### Create an Event Type Custom event type IDs must be prefixed with the tenant name followed by a dot. ```java // Construct the EventTypesClient object as shown above // Define the event type to be created String event_type_name = ""; EventType event_type = new EventType(); event_type.setName(event_type_name); event_type.setTtl(); event_type.setScope(EventType.ScopeEnum.LOCAL); // Define fields for the event type Field field = new Field(); field.setName(""); field.setFilterable(Boolean.TRUE); field.required(Boolean.TRUE); field.setType(Field.TypeEnum.INTEGER); // Add the fields to the event type List field_list = new ArrayList<>(); field_list.add(field); event_type.setFields(field_list); EventType created_event_type = null; try { created_event_type = event_type_client.createEventType(event_type); } catch (MindsphereException e) { // Exception handling } ``` ### Update an Event Type #### Update the Scope of an Event Type ```java // Construct EventTypesClient object as shown above // Define the event type patch to update the event type with EventTypePatch event_type_patch = new EventTypePatch(); event_type_patch.setOp(EventTypePatch.OperationEnum.REPLACE); event_type_patch.setPath("/scope"); event_type_patch.setValue(ScopeEnum.GLOBAL.getValue()); EventType updated_event_type = null; try { updated_event_type = event_type_client.updateEventType("", "", event_type_patch); } catch (MindsphereException e) { // Exception Handling } ``` #### Add a Field to an Event Type ```java // Construct EventTypesClient object as shown above // Create the field to be added Field new_field = new Field(); new_field.setName(""); new_field.setUpdatable(Boolean.FALSE); new_field.setType(Field.TypeEnum.STRING); // Define the event type patch to update the event type with EventTypePatch event_type_patch = new EventTypePatch(); event_type_patch.setOp(EventTypePatch.OperationEnum.ADD); event_type_patch.setPath(EventManagementConstants.ADD_NEW_FIELD); ObjectMapper object_mapper = new ObjectMapper(); String field_as_string = object_mapper.writeValueAsString(field) event_type_patch.setValue(field_as_string); EventType updated_event_type = null; try { updated_event_type = event_type_client.updateEventType("", "", event_type_patch); } catch (MindsphereException e) { // Exception Handling } ``` Alternatively, use the `addNewFieldToEventType` method. ```java //Construct EventTypesClient object as shown above // Create the field to be added Field new_field = new Field(); new_field.setName(""); new_field.setUpdatable(Boolean.FALSE); new_field.setType(Field.TypeEnum.STRING); EventType updated_event_type = null; try { updated_event_type = event_type_client.addNewFieldToEventType(eventType.getId(), "", new_field); } catch (MindsphereException e) { // Exception Handling } ``` #### Make a Field of an Event Type Mandatory ```java // Construct EventTypesClient object as shown above // Define the event type patch to update the event type with EventTypePatch patch = new EventTypePatch(); patch.setOp(EventTypePatch.OperationEnum.REPLACE); patch.setPath("/fields//required"); patch.setValue(Boolean.FALSE); EventType updatedEventType = null; try { updatedEventType = event_type_client.updateEventType("", "", eventTypePatch); } catch (MindsphereException e) { // Exception Handling } ``` ### Read an Event Type by ID ```java //Construct EventTypesClient object as shown above EventType event_type = null; try { event_type = event_type_client.getEventTypesById("", ""); } catch (MindsphereException e) { // Exception Handling } ``` ### Delete an Event Type by ID ```java // Construct the EventTypesClient object as shown above try { event_type_client.deleteEventType("", ""); } catch (MindsphereException ex) { // Exception Handling } ``` ## Event Operations The Events client creates, reads and updates standard and custom events. Client name: [EventsClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/eventmanagement/apiclient/EventsClient.html) ### List all Events Note By default, the response is paginated with 20 elements per page. Attention The `getEvents` method returns a paginated list of `BaseEvent` objects. These objects must be typecasted to be used as `MindsphereStandardEvent` or `CustomEvent` as shown below. #### List all Standard Events ```java // Construct the EventsClient object EventsClient eventClient = EventsClient.builder() .mindsphereCredentials() .restClientConfig() .build(); Events events = null; try { events = eventClient.getEvents(, , "", "", "", history); if (events != null && events.getEmbedded() != null && events.getEmbedded().getEvents() != null && !events.getEmbedded().getEvents().isEmpty()) { List eventList = events.getEmbedded().getEvents(); // Typecast the list object to obtain a MindsphereStandardEvent object MindsphereStandardEvent standard_event = eventList.get(0); } } catch (MindsphereException ex) { // Exception Handling } ``` #### List all Events of a Specific Event Type For obtaining events of specific (custom) event type, a filter for the respective event type ID must be defined, e.g. for event type ID `e5e6460e-60da-4cc9-b4b7-a7af3cc9fca6`: ```java String filter = "{\"typeId\":\"e5e6460e-60da-4cc9-b4b7-a7af3cc9fca6\"}"; ``` Sample implementation: ```java // Construct the EventsClient object as shown above Events events = null; try { String filter = "{\"type_id\":\"\"}"; events = eventClient.getEvents(, , filter, "", "", history); if (events != null && events.getEmbedded() != null && events.getEmbedded().getEvents() != null && !events.getEmbedded().getEvents().isEmpty()) { List eventList = events.getEmbedded().getEvents(); CustomEvent custom_event = (CustomEvent)eventList.get(0); } } catch (MindsphereException ex) { // Exception Handling } ``` ### Create a Standard Event ```java // Construct the EventsClient object as shown above // Construct the MindSphereStandardEvent object to be created MindSphereStandardEvent standard_event = new MindSphereStandardEventt(); // Set the base properties standard_event.setCorrelationId(""); standard_event.setEntityId(""); standard_event.setTimestamp(""); // Set the standard event properties standard_event.setDescription(""); standard_event.setSeverity(); standard_event.setCode(""); standard_event.setAcknowledged(); MindSphereStandardEvent created_standard_event = null; try { created_standard_event = eventClient.createStandardEvent(standard_event); } catch (MindsphereException ex) { // Exception Handling } ``` Alternatively, provide the standard event in JSON format. ```java String standard_event_as_json = "{" + "\"correlationId\":\"\"," + "\"timestamp\":\"\"," + "\"severity\":," + "\"description\":\"\"," + "\"code":\"\"," + "\"source\":\"source\"," + "\"acknowledged\":\"" + "}"; MindSphereStandardEvent created_standard_event = null; try { created_standard_event = (MindSphereStandardEvent) eventClient.createEvent(standard_event_as_json); } catch (MindsphereException ex) { // Exception Handling } ``` ### Create a Custom Event Custom events are instantiated from a user-defined event type. ```java // Construct the EventsClient object as shown above // Construct the CustomEvent object to be created CustomEvent custom_event = new CustomEvent(); // Set the base properties custom_event.setCorrelationId(""); custom_event.setEntityId(""); custom_event.setTimestamp(""); custom_event.setTypeId(""); // Set values for custom fields as hash maps using a key and value pair custom_event.getFields().put("", ""); custom_event.getFields().put("", ""); CustomEvent created_custom_event = null; try { created_custom_event = eventClient.createCustomEvent(custom_event); } catch (MindsphereException ex) { // Exception Handling } ``` Alternatively, provide the custom event as in JSON format. ```java String custom_event_as_json = "{" + "\"typeId\":\"\"," + "\"correlationId\":\"\"," + "\"timestamp\":\"\"," + "\"entityId\":\"\"," + "\"\":\"custom_field_value1\"," + "\"\":\"custom_field_value2\"," + "}"; CustomEvent created_custom_event = null; try { created_custom_event = (CustomEvent)eventClient.createEvent(custom_event_as_json); } catch (MindsphereException ex) { // Exception Handling } ``` ### Get an Event by ID ```java // Construct the EventsClient object as shown above BaseEvent event = null; try { event = eventClient.getEventById(""); // Check the type of the retrieved event if (event != null && event instanceof MindsphereStandardEvent) { // Process the retrieved standard event MindsphereStandardEvent standard_event = (MindsphereStandardEvent) event; } else if (event != null && event instanceof CustomEvent) { // Process the retrieved custom event CustomEvent custom_event = (CustomEvent) event; } } catch (MindsphereException ex) { // Exception Handling } ``` ### Update an Event Updates the event with the specified ID. ```java // Construct the EventsClient object as shown above BaseEvent event = null; try { event = eventClient.updateEvents("", "", ""); // Check the type of the retrieved event as shown above for further processing } catch (MindsphereException ex) { // Exception Handling } ``` ### Update a Standard Event Updates the standard event with the specified ID. ```java // Construct the EventsClient object as shown above MindSphereStandardEvent updated_standard_event = null; try { updated_standard_event = eventClient.updateStandardEvent("", "", ""); } catch (MindsphereException ex) { // Exception Handling } ``` ### Update a Custom Event Updates the custom event with the specified ID. ```java // Construct the EventsClient object as shown above CustomEvent custom_event = null; try { custom_event = eventClient.updateCustomEvent("", "", ""); } catch (MindsphereException ex) { // Exception Handling } ``` ## Event Job Operations The Jobs client creates and retrieves jobs for creating or deleting events. Client name: [JobsClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/eventmanagement/apiclient/JobsClient.html) ### Create a Job to Create Events #### Create a Job to Create Custom Events ```java // Construct a JobsClient object JobsClient jobs_client = JobsClient.builder() .restClientConfig() .build(); CustomEvent custom_event = new CustomEvent(); custom_event.setTypeId(eventTypeId1); CustomEvent custom_event2 = new CustomEvent(); custom_event2.setTypeId(eventTypeId2); List event_list = new ArrayList<>(); event_list.add(custom_event); event_list.add(custom_event2); CreateEventsJob create_events_job = new CreateEventsJob(); create_events_job.setEvents(event_list); JobResource job_resource = null; try { job_resource = jobs_client.createCreateEventsJob(create_events_job); } catch (MindsphereException e) { // Exception handling } ``` #### Create a Job to Create Standard Events ```java MindSphereStandardEvent standard_event = new MindSphereStandardEvent(); MindSphereStandardEvent standard_event2 = new MindSphereStandardEvent(); List eventList = new ArrayList<>(); eventList.add(standard_event); eventList.add(standard_event2); CreateEventsJob create_events_job = new CreateEventsJob(); create_events_job.setEvents(eventList); JobResource job_resource = null; try { job_resource = jobs_client.createCreateEventsJob(create_events_job); } catch (MindsphereException e) { // Exception handling } ``` ### Get a Job to Create Events by its ID ```java //Construct JobsClient object as shown above String job_id = ""; CreateJobResource create_job_resource = null; try { create_job_resource = jobs_client.getCreateEventsJobById(job_id); } catch (MindsphereException e) { // Exception handling } ``` ### Create a Job to Delete Events Create a job to delete events which satisfy query parameters. The query parameters are listed in the filter method of the [DeleteEventsJob](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/eventmanagement/model/DeleteEventsJob.html) class. ```java // Create a map of events to be deleted HashMap filter = new HashMap<>(); filter.put("typeId", "com.siemens.mindsphere.eventmgmt.event.type.MindSphereStandardEvent"); // Create the DeleteEventsJob instance DeleteEventsJob delete_events_job = new DeleteEventsJob(); delete_events_job.setFilter(filter); JobResource job_event = null; try { job_event = jobs_client.createDeleteEventsJob(delete_events_job); } catch (MindsphereException e) { // Exception handling } ``` ### Get a Job to Delete Events by its ID ```java //Construct JobsClient object as shown above String job_id = ""; DeleteJobResource delete_job_resource = null; try { delete_job_resource = jobs_client.getDeleteEventsJobById(job_id); } catch (MindsphereException e) { // Exception handling } ``` # Integrated Data Lake Client for Java ## Introduction The Integrated Data Lake Java client allows you to perform operations provided by Integrated Data Lake Service. Further implementation of the Integrated Data Lake SDK library has been shown in a sample project that you can download and test in local or on Insights Hub application. Please refer to this repository: [industrial-iot-java-sdk-examples](https://github.com/mindsphere/mindsphere-java-sdk-examples) For more information about the service, refer to [Integrated Data Lake Service](../../../apis/iot-integrated-data-lake/api-integrated-data-lake-overview.html). Hint Placeholders in the following samples are indicated by angular brackets `< >`. ## Integrated Data Lake Operations Client Name: [ObjectOperationsClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/integrateddatalake/apiclient/ObjectOperationsClient.html) ### Query objects - Retrieve information of existing objects. In case a folder is specified as input path, all child objects within the folder are returned. Objects in sub-folders of the given folder are not returned. In case a file is specified as input path, only the file is returned. Only the objects of one tenant or subtenant are returned. In case a tenant wants to retrieve information of a subtenant objects, the subtenantId query parameter must be used. ```java // Construct the ObjectOperationsClient object ObjectOperationsClient objectOperationsClient = ObjectOperationsClient.builder() .mindsphereCredentials() .restClientConfig() .build(); QueryObjectsRequest requestObj = new QueryObjectsRequest(); try { ObjectListResponse response = objectOperationsClient.queryObjects(requestObj); } catch (MindsphereException e) { // Exception handling } ``` ### Delete an object - Delete the object located at specified path. ```java DeleteObjectRequest requestObj = new DeleteObjectRequest(); requestObj.setPath("data/ten=dide2/e2eMetaTag-DONOTDELETE.txt"); try { objectOperationsClient.deleteObject(requestObj); } catch (MindsphereException e) { // Exception handling } ``` ### Delete multiple objects in bulk - Create asynchronous job to delete multiple objects for the specified paths. If any of the given paths doesn't exist then job will return status as COMPLETED_WITH_ERRORS. Jobs which are completed with or without errors would be deleted periodically.Data once deleted cannot be recovered. ```java CreateDeleteObjectsJobRequest requestObj = new CreateDeleteObjectsJobRequest(); DeleteObjectsJobRequest deleteObjectsJob = new DeleteObjectsJobRequest(); List objects = new ArrayList(); DeleteObjectsJobRequestObjects deleteObjectsJobRequestObjects = new DeleteObjectsJobRequestObjects(); deleteObjectsJobRequestObjects.setPath("data/ten=dide2/e2eMetaTag-DONOTDELETE.txt"); objects.add(deleteObjectsJobRequestObjects); deleteObjectsJob.setObjects(objects); requestObj.setDeleteObjectsJob(deleteObjectsJob); try { DeleteObjectsJobResponse response =objectOperationsClient.createDeleteObjectsJob(requestObj); } catch (MindsphereException e) { // Exception handling } ``` ### Fetch details of all the Delete Objects Jobs - Create asynchronous job to delete multiple objects for the specified paths. If any of the given paths doesn't exist then job will return the status as COMPLETED_WITH_ERRORS. Jobs which are completed with or without errors would be deleted periodically.Data once deleted cannot be recovered. ```java GetAllDeleteObjectsJobRequest requestObj = new GetAllDeleteObjectsJobRequest(); try { DeleteObjectsJobList response = objectOperationsClient.getAllDeleteObjectsJob(requestObj); } catch (MindsphereException e) { // Exception handling } ``` ### Fetch status and details of Delete Objects Job - Retrieve information of given Delete Objects Job. Only the objects of one tenant or subtenant are returned. In case a tenant wants to retrieve information on a subtenant's objects, the subtenantId query parameter must be used. ```java GetDeleteObjectsJobRequest requestObj = new GetDeleteObjectsJobRequest(); requestObj.setId("075aad4f-b50c-423e-a934-b6408306fbfb"); try { DeleteObjectsJobResponse response = objectOperationsClient.getDeleteObjectsJob(requestObj); } catch (MindsphereException e) { // Exception handling } ``` ### Fetch errors of Delete Objects Job - Retrieve errors of given Delete Objects Job. This endpoint should be called only when /deleteObjectsJobs/{id} returns the status as COMPLETED_WITH_ERRORS. Only the objects of one tenant or subtenant are returned. In case a tenant wants to retrieve information on a subtenant objects, the subtenantId query parameter must be used. ```java GetDeleteObjectsJobErrorsRequest requestObj = new GetDeleteObjectsJobErrorsRequest(); requestObj.setId("a615f4eb-3de6-4fda-95b3-76524c5ffdb7"); try { DeleteObjectsJobErrorDetailsResponse response = objectOperationsClient.getDeleteObjectsJobErrors(requestObj); } catch (MindsphereException e) { // Exception handling } ``` Client Name: [ObjectEventSubscriptionOperationsClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/integrateddatalake/apiclient/ObjectEventSubscriptionOperationsClient.html) ### List event subscriptions for the tenant or subtenant - List object event subscriptions for the tenant or subtenant. If requester is tenant, all the subscriptions for the tenant as well as its all subtenants are returned. If requester is a subtenant, all the subscriptions for the subtenant are returned. If tenant wants to filter results for a particular subtenant, filter query parameter subtenantId can be used. This filter query parameter is applicable only if the requester is tenant. ```java // Construct the ObjectEventSubscriptionOperationsClient object ObjectEventSubscriptionOperationsClient eventSubscriptionOperationsClient = ObjectEventSubscriptionOperationsClient.builder() .mindsphereCredentials() .restClientConfig() .build(); QueryObjectEventSubscriptionsRequest requestObj = new QueryObjectEventSubscriptionsRequest(); try { SubscriptionListResource response = eventSubscriptionOperationsClient.queryObjectEventSubscriptions(requestObj); } catch (MindsphereException e) { // Exception handling } ``` ### Create object event subscription - Allows users to subscribe for event notifications generated when the objects of a tenant or subtenant are created, updated, or deleted. Multiple subscriptions for the same path can be created when each has a different destination. Similarly, multiple subscriptions for the same destination can be created when each has a different path. Maximum 15 subscriptions can be created for a tenant or for a subtenant. Path in request payload should be upto folders and not upto object e.g. "myfolder/mysubfolder". Notification Content Based on the configured subscriptions,event notification messages are published to the destination. The event notification content is formatted in JSON according to this example. If object operation happened in subtenant folder, both tenantId and subtenantId will be part of the message. If object operation happened in tenant folder, only tenantId will be part of the message. ```java CreateObjectEventSubscriptionRequest requestObj = new CreateObjectEventSubscriptionRequest(); Subscription subscription = new Subscription(); subscription.setDestination("aws-sns://arn:aws:sns:eu-central-1:210761511742:dl-test"); subscription.setPath("myfolder/mysubfolder"); requestObj.setSubscription(subscription); try { SubscriptionResponse response = eventSubscriptionOperationsClient.createObjectEventSubscription(requestObj); } catch (MindsphereException e) { // Exception handling } ``` ### Update object event subscription by id - Update object event subscription. ```java PatchObjectEventSubscriptionRequest requestObj = new PatchObjectEventSubscriptionRequest(); requestObj.setId("1d8d741f-645f-4b9f-a913-0feaee4381d0"); requestObj.setIfMatch(2); SubscriptionUpdate subscription = new SubscriptionUpdate(); subscription.setDestination("aws-sns://arn:aws:sns:eu-central-1:210761511742:dl-test"); subscription.setPath("myfolder/mysubfolder"); requestObj.setSubscription(subscription); try { SubscriptionResponse response = eventSubscriptionOperationsClient.patchObjectEventSubscription(requestObj); } catch (MindsphereException e) { // Exception handling } ``` ### Read object event subscription by id - Read object event subscription for the tenant. ```java RetrieveObjectEventSubscriptionRequest requestObj = new RetrieveObjectEventSubscriptionRequest(); try { SubscriptionResponse response = eventSubscriptionOperationsClient.retrieveObjectEventSubscription(requestObj); } catch (MindsphereException e) { // Exception handling } ``` ### Delete object event subscription by id - Delete object event subscription (Unsubscribe). ```java DeleteObjectEventSubscriptionRequest requestObj = new DeleteObjectEventSubscriptionRequest(); requestObj.setId("1d8d741f-645f-4b9f-a913-0feaee4381d0"); requestObj.setIfMatch(3); try { eventSubscriptionOperationsClient.deleteObjectEventSubscription(requestObj); } catch (MindsphereException e) { // Exception handling } ``` Client Name: [ObjectAccessOperationsClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/integrateddatalake/apiclient/ObjectAccessOperationsClient.html) ### Generate signed URLs to upload an object - Generate signed URLs to upload one or more objects. - Signed URL response for upload ```java // Construct the ObjectOperationsClient object ObjectAccessOperationsClient operationsClient = ObjectAccessOperationsClient.builder() .mindsphereCredentials() .restClientConfig() .build(); GenerateUploadObjectUrlsRequest requestObj = new GenerateUploadObjectUrlsRequest(); GenerateUrlPayload generateUrlPayload = new GenerateUrlPayload(); Path path = new Path(); path.setPath("myfolder/mysubfolder/myobject.objext"); Paths paths = new Paths(); paths.add(path); generateUrlPayload.setPaths(paths); requestObj.setGenerateUrlPayload(generateUrlPayload); try { SignedUrlResponse response = operationsClient.generateUploadObjectUrls(requestObj); } catch (MindsphereException e) { // Exception handling } ``` ### Generate signed URLs to download an object - Generate signed URLs to download single object. - Generate signed URLs to download one or more objects. - Signed URL response for download ```java GenerateDownloadObjectUrlsRequest requestObj = new GenerateDownloadObjectUrlsRequest(); GenerateUrlPayload generateUrlPayload = new GenerateUrlPayload(); Path path = new Path(); path.setPath("myfolder/mysubfolder/myobject.objext"); Paths paths = new Paths(); paths.add(path); generateUrlPayload.setPaths(paths); requestObj.setGenerateUrlPayload(generateUrlPayload); try { SignedUrlResponse response = operationsClient.generateDownloadObjectUrls(requestObj); } catch (MindsphereException e) { // Exception handling } ``` ### Get a list of cross accounts - Get list of cross accounts on which access was given. - If requester is tenant, all the cross account for the tenant as well as its all subtenants are returned. - If requester is a subtenant, all the cross accounts for the subtenant are returned. - If tenant wants to filter results for a particular subtenant, filter query parameter subtenantId can be used. This filter query parameter is applicable only if the requester is tenant. ```java ListCrossAccountRequest requestObj = new ListCrossAccountRequest(); try { CrossAccountListResource response = operationsClient.listCrossAccount(requestObj); } catch (MindsphereException e) { // Exception handling } ``` ### Create a cross account - Create a cross account on which access needs to be given for paths. - If, in request body subtenant is denoted as , it is a special cross account. In this case, on this cross account, tenant implicitly gets Read Only access to the storage account's root level data path for itself and for all its subtenants. For this case, the authorization token should be that of the tenant. ```java CreateCrossAccountRequest requestObj = new CreateCrossAccountRequest(); CrossAccountRequest crossAccountRequest = new CrossAccountRequest(); crossAccountRequest.setAccessorAccountId("565037524705"); crossAccountRequest.setName("UI Account"); crossAccountRequest.setDescription("Test Accontttt"); requestObj.setCrossAccountRequest(crossAccountRequest); try { CrossAccount response = operationsClient.createCrossAccount(requestObj); } catch (MindsphereException e) { // Exception handling } ``` ### Get details of selected cross account - Get details of selected cross account. ```java GetCrossAccountRequest requestObj = new GetCrossAccountRequest(); requestObj.setId("d5395d2b-524e-4a28-b100-2138b6f078f4"); try { CrossAccount response = operationsClient.getCrossAccount(requestObj); } catch (MindsphereException e) { // Exception handling } ``` ### Update a cross account - Update a cross account on which access needs to be managed. ```java UpdateCrossAccountRequest requestObj = new UpdateCrossAccountRequest(); CrossAccountUpdateRequest crossAccountUpdateRequest = new CrossAccountUpdateRequest(); crossAccountUpdateRequest.setName("abc"); crossAccountUpdateRequest.setDescription("rgreg"); requestObj.setCrossAccountRequest(crossAccountUpdateRequest); requestObj.setId("f2904dfa-6784-423a-8573-41a77219825e"); requestObj.setIfMatch(3); try { CrossAccount response = operationsClient.updateCrossAccount(requestObj); } catch (MindsphereException e) { // Exception handling } ``` ### Delete cross account - Delete cross account and corresponding accesses. ```java DeleteCrossAccountRequest requestObj = new DeleteCrossAccountRequest(); requestObj.setId("e628bd56-8b46-4a01-bc82-82aa9ec2a880"); requestObj.setIfMatch(2); try { operationsClient.deleteCrossAccount(requestObj); } catch (MindsphereException e) { // Exception handling } ``` ### Get a list of cross account accesses This section shows two options for achieving the following behavior: - Get list of cross account accesses. ```java ListCrossAccountAccessRequest requestObj = new ListCrossAccountAccessRequest(); requestObj.setId("8a1dce46-30db-4f80-ab78-9740dcdb6386"); try { CrossAccountAccessListResource response = operationsClient.listCrossAccountAccess(requestObj); } catch (MindsphereException e) { // Exception handling } ``` ### Create a cross account access - Create a cross account access. Create a cross account access. If the cross account is created for tenant and all subtenants (\*\*), then this operation is implicitly handled and not allowed through API. Maximum 5 cross account accesses can be created with ENABLED status. Maximum 10 cross accounts can be created with DISABLED status. ```java CreateCrossAccountAccessRequest requestObj = new CreateCrossAccountAccessRequest(); requestObj.setId("8a1dce46-30db-4f80-ab78-9740dcdb6386"); CrossAccountAccessRequest crossAccountAccessRequest = new CrossAccountAccessRequest(); crossAccountAccessRequest.setDescription("stringefewfup"); crossAccountAccessRequest.setPath("myfolder/mysubfolder"); crossAccountAccessRequest.setPermission(Permission.DELETE); crossAccountAccessRequest.setStatus(StatusEnum.DISABLED); requestObj.setCrossAccountAccessRequest(crossAccountAccessRequest); try { CrossAccountAccess response = operationsClient.createCrossAccountAccess(requestObj); } catch (MindsphereException e) { // Exception handling } ``` ### Get a cross account access - Get a selected access for a selected cross account. ```java GetCrossAccountAccessRequest requestObj = new GetCrossAccountAccessRequest(); requestObj.setId("8a1dce46-30db-4f80-ab78-9740dcdb6386"); requestObj.setAccessId("c24e4208-ff7c-4909-aee7-214dba800446"); try { CrossAccountAccess response = operationsClient.getCrossAccountAccess(requestObj); } catch (MindsphereException e) { // Exception handling } ``` ### Update a cross account access - Update a cross account access. This operation is not allowed when the cross account is created for special case of tenant and all subtenants (\*\*). Maximum 5 cross account accesses can be present with ENABLED status. Maximum 10 cross accounts can be present with DISABLED status. ```java UpdateCrossAccountAccessRequest requestObj = new UpdateCrossAccountAccessRequest(); requestObj.setId("2ad2371a-6ce9-4474-bb4a-25cfe1f39da3"); requestObj.setAccessId("b45993e8-6984-48ac-a0fc-953ec53a23e7"); CrossAccountAccessRequest crossAccountAccessRequest = new CrossAccountAccessRequest(); crossAccountAccessRequest.setDescription("update"); crossAccountAccessRequest.setPath("myfolder/mysubfolder"); crossAccountAccessRequest.setPermission(Permission.READ); crossAccountAccessRequest.setStatus(StatusEnum.DISABLED); requestObj.setCrossAccountAccessRequest(crossAccountAccessRequest); requestObj.setIfMatch(10); try { CrossAccountAccess response = operationsClient.updateCrossAccountAccess(requestObj); } catch (MindsphereException e) { // Exception handling } ``` ### Delete a cross account access If the cross account is created for tenant and all subtenants (\*\*), then this operation is implicitly handled by deletion of cross account and not allowed through API. ```java DeleteCrossAccountAccessRequest requestObj = new DeleteCrossAccountAccessRequest(); requestObj.setId("8a1dce46-30db-4f80-ab78-9740dcdb6386"); requestObj.setAccessId("883b6eeb-ea39-4967-b288-0658400906e6"); requestObj.setIfMatch(1); try { operationsClient.deleteCrossAccountAccess(requestObj); } catch (MindsphereException e) { // Exception handling } ``` Client Name: [ObjectOperationsWithAccessTokenClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/integrateddatalake/apiclient/ObjectOperationsWithAccessTokenClient.html) ### List all folders having write permission This API can be accessed by tenantadmin to list all write permission folders including that of subtenants. Subtenant can access this API to get list write permission folders owned by subtenant. Size parameter value should not be more than 1000. ```java // Construct the ObjectOperationsWithAccessTokenClient object ObjectOperationsWithAccessTokenClient accessTokenClient = ObjectOperationsWithAccessTokenClient.builder() .mindsphereCredentials() .restClientConfig() .build(); ListAccessTokenPermissionsRequest requestObj = new ListAccessTokenPermissionsRequest(); try { AccessTokenPermissionResources response = accessTokenClient.listAccessTokenPermissions(requestObj); } catch (MindsphereException e) { // Exception handling } ``` ### Allows to give write permission on folder/path - Allows users (tenant) to give write permission on folder/path. This API can only be accessed by tenantadmin. Tenantadmin can also provide write permission on subtenant folder. Write permission on root folder can also be given, except TSI folder ```java AccessTokenPermissionsRequest requestObj = new AccessTokenPermissionsRequest(); AccessTokenPermissionRequest accessTokenPermissionRequest = new AccessTokenPermissionRequest(); accessTokenPermissionRequest.setPath("customFolder"); ccessTokenPermissionRequest.setPermission(AccessPermission.WRITE); requestObj.setWritePathPayload(accessTokenPermissionRequest); try { AccessTokenPermissionResource response = accessTokenClient.accessTokenPermissions(requestObj); } catch (MindsphereException e) { // Exception handling } ``` ### Details of the write folder request for the given id This API can be accessed by tenantadmin to get details of the request including that for subtenants. Subtenant can access this API, to get details of the request belongs to their write folder. ```java GetAccessTokenPermissionsRequest requestObj = new GetAccessTokenPermissionsRequest(); requestObj.setId("672eaace-60fd-4b90-865d-795b981e5ac8"); try { AccessTokenPermissionResource response = accessTokenClient.getAccessTokenPermissions(requestObj); } catch (MindsphereException e) { // Exception handling } ``` ### Generate AWS STS token - Allows the users to request temporary, limited-privilege AWS credentials to get read-only or write-only access on the URI returned in the response. - Read permission will always be on the root level. - Path field is optional for READ permission. If value for path is not provided then it will be considered on root level - Ensure to enable write access on the path before requesting token with write permission. - Write access can be enabled using POST /accessTokenPermissions endpoint. - An access token requested for a given path also automatically gives access to all subpaths of the path. For example, if an access token is requested for path /a and there are subpaths /a/b and /a/b/c, the token allows to access those too. - An access token with write permissions can only be requested for the paths defined by resource accessTokenPermissions. An access token with read permissions can only be requested for the root path /. ```java GenerateAccessTokenRequest requestObj = new GenerateAccessTokenRequest(); GenerateSTSPayload generateSTSPayload = new GenerateSTSPayload(); requestObj.setStsPayload(generateSTSPayload); try { AccessTokens response = accessTokenClient.generateAccessToken(requestObj); } catch (MindsphereException e) { // Exception handling } ``` ### Delete write permission on folder for the given id This API can be accessed by the tenantadmin only. ```java DeleteAccessTokenPermissionsRequest requestObj = new DeleteAccessTokenPermissionsRequest(); requestObj.setId("4869c9ab-3430-471c-aaec-901d16d0a1a1"); try { accessTokenClient.deleteAccessTokenPermissions(requestObj); } catch (MindsphereException e) { // Exception handling } ``` Client Name: [TimeSeriesBulkImportClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/integrateddatalake/apiclient/TimeSeriesBulkImportClient.html) ### Query all time series bulk import jobs - Query all time series bulk import jobs currently existing, which are owned by the client tenant or subtenant. If requester is tenant, all the import jobs for the tenant as well as its all subtenants are returned. If requester is a subtenant, all the iport jobs for the subtenant are returned. If tenant wants to filter results for a particular subtenant, filter query parameter subtenantId can be used. This filter query parameter is applicable only if the requester is tenant. ```java // Construct the TimeSeriesBulkImportClient object TimeSeriesBulkImportClient accessTokenClient = TimeSeriesBulkImportClient.builder() .mindsphereCredentials() .restClientConfig() .build(); QueryTimeSeriesImportJobsRequest timeSeriesBulkImportClient = new QueryTimeSeriesImportJobsRequest(); try { ImportJobListResource response = timeSeriesBulkImportClient.queryTimeSeriesImportJobs(requestObj); } catch (MindsphereException e) { // Exception handling } ``` ### Create a bulk import job of time series data into data lake The import takes into account time series data from the provided aspects associated to the provided assets, in the given time range. ```java CreateTimeSeriesImportJobRequest requestObj = new CreateTimeSeriesImportJobRequest(); ImportJobRequest importJobRequest = new ImportJobRequest(); importJobRequest.setName("string5"); importJobRequest.setDestination("myfolder/mysubfolder"); importJobRequest.setFrom("2017-12-11T13:36:00.000Z"); importJobRequest.setTo("2018-01-11T13:36:00.000Z"); List aspectNames = new ArrayList(); List assetIds = new ArrayList(); aspectNames.add("string"); assetIds.add("string"); importJobRequest.setAspectNames(aspectNames); importJobRequest.setAssetIds(assetIds); requestObj.setImportJob(importJobRequest); try { ImportJobResponse response = timeSeriesBulkImportClient.createTimeSeriesImportJob(requestObj); } catch (MindsphereException e) { // Exception handling } ``` ### Retrieve details of a time series bulk import job The details are only available once a job is not any longer in status PENDING. ```java RetrieveTimeSeriesImportJobDetailsRequest requestObj = new RetrieveTimeSeriesImportJobDetailsRequest(); requestObj.setId("2faf09913995428b90997f834da2f542"); try { ImportJobDetails response = timeSeriesBulkImportClient.retrieveTimeSeriesImportJobDetails(requestObj); } catch (MindsphereException e) { // Exception handling } ``` ### Delete time series bulk import job by id Deletes the completed time series bulk import job. ```java DeleteTimeSeriesImportJobRequest requestObj = new DeleteTimeSeriesImportJobRequest(); requestObj.setId("9842e5ebf37943328d97c33b6814a821"); try { timeSeriesBulkImportClient.deleteTimeSeriesImportJob(requestObj); } catch (MindsphereException e) { // Exception handling } ``` ### Retrieve status of time series bulk import job Retrieves the status of time series bulk import job. ```java RetrieveTimeSeriesImportJobRequest requestObj = new RetrieveTimeSeriesImportJobRequest(); requestObj.setId("2faf09913995428b90997f834da2f542"); try { ImportJobResponse response = timeSeriesBulkImportClient.retrieveTimeSeriesImportJob(requestObj); } catch (MindsphereException e) { // Exception handling } ``` Client Name: [ObjectsMetadataCatalogOperationsClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/integrateddatalake/apiclient/ObjectsMetadataCatalogOperationsClient.html) ### Create/Update Metadata for the object - Create/Update metadata. If tenant is operating for a subtenant object, he should specify subtenant in request query parameter. If path contains special characters which require URL encoding, it should be done as appropriate. Maximum 8 tags are allowed, where each tag can be maximum 128 characters. ```java // Construct the ObjectOperationsClient object ObjectsMetadataCatalogOperationsClient oCatalogOperationsClient = ObjectsMetadataCatalogOperationsClient.builder() .mindsphereCredentials() .restClientConfig() .build(); CreateOrUpdateObjectMetadataRequest requestObj = new CreateOrUpdateObjectMetadataRequest(); requestObj.setPath("myfolder/mysubfolder/myobject.objext"); Metadata metadata = new Metadata(); List tags = new ArrayList(); tags.add("tag1"); tags.add("tag2"); metadata.setTags(tags); requestObj.setMetadata(metadata); try { oCatalogOperationsClient.createOrUpdateObjectMetadata(requestObj); } catch (MindsphereException e) { // Exception handling } ``` ### Get Metadata for the object If tenant is operating for a subtenant object, he should specify subtenant in request query parameter. If path contains special characters which require URL encoding, it should be done as appropriate. ```java RetrieveObjectMetadataRequest requestObj = new RetrieveObjectMetadataRequest(); requestObj.setPath("myfolder/mysubfolder/myobject.objext"); try { ObjectMetaDataResponse response = oCatalogOperationsClient.retrieveObjectMetadata(requestObj); } catch (MindsphereException e) { // Exception handling } ``` # IoT TS Aggregates Client for Java ## Introduction The IoT TS Aggregates Java client allows you to query aggregated time series data. Refer to [IoT TS Aggregates](../../../apis/iot-iottsaggregates/api-iottsaggregates-overview.html) for more information about the service. Further implementation of the TS Aggregate SDK library has been shown in a sample project that you can download and test in local or on Insights Hub application. Please refer to this repository: [industrial-iot-java-sdk-examples](https://github.com/mindsphere/mindsphere-java-sdk-examples) Hint In the IoT context, *assets* are referred to as *entity* and *aspects* as *propertyset*.\ Placeholders in the following samples are indicated by angular brackets `< >`. ## TS Aggregates Operations Client name: [AggregatesClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/tsaggregates/apiclient/AggregatesClient.html) ### Read Aggregated Time Series This section shows two options for reading aggregated time series data of a specific aspect of an asset from the specified time range. This feature is only available if aspect data is available within the specified time range. Note A query can only return up to 200 aggregate intervals in one response.\ Aggregates can only be created from up to 5,000 raw time series entries. ```java // Construct the AggregatesClient object AggregatesClient aggregate_client = AggregatesClient.builder() .mindsphereCredentials() .restClientConfig() .build(); List aggregate_list = null; try { aggregate_list = aggregate_client.getAggregateTimeseries(, , , , , , ); List aggregate_list = null; try { aggregate_list = aggregate_client.getAggregateTimeseries(request_object); } catch (MindsphereException e) { // Exception handling } ``` # IOT Bulk Client for Java ## Introduction The IoT TS Bulk Java client allows you to use IoT TS Bulk APIs while developing applications for Insights Hub. Refer to [IoT TS Bulk Service](../../../apis/iot-iottsbulk/api-iottsbulk-overview.html) for more information about the service. Further implementation of the TS Bulk SDK library has been shown in a sample project that you can download and test in local or on Insights Hub application. Please refer to this repository: [industrial-iot-java-sdk-examples](https://github.com/mindsphere/mindsphere-java-sdk-examples) Hint In the IoT context, *assets* are referred to as *entity* and *aspects* as *propertyset*.\ Placeholders in the following samples are indicated by angular brackets `< >`. ## IOT Bulk Operations Client Name: [BulkImportOperationsClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/iotbulk/apiclient/BulkImportOperationsClient.html) ### Create bulk import job for importing time series data - Creates an import job resource to asynchronously import IoT time series data from files uploaded through IoT File Service. After successful creation of an import job, the provided file contents are validated and imported in the background. The status of a job can be retrieved using the returned job ID. Note that in case of validation errors occurring during or after job creation, no time series data is imported from any of the provided files. - Restrictions: Currently only one asset-aspect (entity-property set) combination can be specified as target of the import. Data for performance assets (entities) must be older than 30 minutes in order to be imported, while for simulation assets (entities) no restriction on minimum age exists. In case of simulation assets (entities), all data must be within the same hour. In case of performance assets (entities), all data must be within the same day. The overall size of the files used to import data for one asset-aspect (entity-property set) combination is limited: - For simulation assets (entities), a maximum of 350 MB per hour is allowed - For performance assets (entities), a maximum of 1 GB per day is allowed Note that hour and day intervals are fixed with respect to UTC time hours and days. A maximum of 100 files can be specified per request. ```java // Construct the PackageProvisioningClient object BulkImportOperationsClient bulkImportOperationsClient = BulkImportOperationsClient.builder().restClientConfig(config).build(); CreateImportJobRequest requestObject = new CreateImportJobRequest(); BulkImportInput bulkImportInput = new BulkImportInput(); List listData = new ArrayList(); Data data = new Data(); data.setEntity("5908ae5c5e4f4e18b0be58cd21ee675f"); List timeseriesFiles = new ArrayList(); data.setPropertySetName("test_2020_11_11"); FileInfo fileInfo = new FileInfo(); fileInfo.setFilePath("test1112.json"); fileInfo.setFrom("2020-06-16T02:00:00.01Z"); fileInfo.setTo("2020-06-16T02:00:00.30Z"); timeseriesFiles.add(fileInfo); data.setTimeseriesFiles(timeseriesFiles); listData.add(data); bulkImportInput.setData(listData); requestObject.setBulkImportInput(bulkImportInput); try { JobStatus response = bulkImportOperationsClient.createImportJob(requestObject); } catch (MindsphereException e) { // Exception handling } ``` ### Retrieve status of bulk import job - Retrieve status of bulk import job. ```java RetrieveImportJobRequest requestObject = new RetrieveImportJobRequest(); requestObject.setId(("4e37b10ad4774b8bb391209a9d18f1b6")); try { JobStatus response = bulkImportOperationsClient.retrieveImportJob(requestObject); } catch (MindsphereException e) { // Exception handling } ``` Client Name: [ReadOperationsClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/iotbulk/apiclient/ReadOperationsClient.html) ### Retrieve time series data - Retrieve time series data for a single asset (entity) and aspect (property set). Returns data for the specified time range. Returns only the selected properties if 'select' parameter is used. The maximum number of time series data items returned per request is defined by parameter. In case more time series data items are present in the requested time range, only a subset of data items will be returned. In that case response property nextRecord contains the request URL to fetch the next set of time series data items, by increasing the parameter accordingly. ```java // Construct the PackageProvisioningClient object ReadOperationsClient readOperationsClient = ReadOperationsClient.builder().restClientConfig(config).build(); RetrieveTimeseriesRequest requestObject = new RetrieveTimeseriesRequest(); requestObject.setEntity("4ab1c8c37a3c47c48447057766ba4107"); requestObject.setPropertySetName("testaspect1011"); requestObject.setFrom("2020-06-16T02:00:00.01Z"); requestObject.setTo("2020-06-16T02:00:00.30Z"); try { TimeSeries response = readOperationsClient.retrieveTimeseries(requestObject); } catch (MindsphereException e) { // Exception handling } ``` # IoT File Client for Java ## Introduction The IoT File Java client allows you to manage files related to assets. Refer to [IoT File Service](../../../apis/iot-iotfile/api-iotfile-overview.html) for more information about the service. Further implementation of the IoT file client SDK library has been shown in a sample project that you can download and test in local or on Insights Hub application. Please refer to this repository: [industrial-iot-java-sdk-examples](https://github.com/mindsphere/mindsphere-java-sdk-examples) Hint In the IoT context, *assets* are referred to as *entity* and *aspects* as *propertyset*.\ Placeholders in the following samples are indicated by angular brackets `< >`. ## File Operations Client name: [FileServiceClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/iotfileservices/apiclient/FileServiceClient.html) ### Create or Update a File This section shows two options for creating or updating a file with the provided content for an asset in the specified path. ```java // Construct the FileServiceClient object FileServiceClient file_service_client = FileServiceClient.builder() .mindsphereCredentials() .restClientConfig() .build(); try { file_service_client.putFile(, , , , , , ); } catch (MindsphereException e) { // Exception handling } ``` Alternatively, use the [`PutFileRequest`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/iotfileservices/model/PutFileRequest.html) model. ```java // Construct the FileServiceClient object as shown above PutFileRequest request_object = new PutFileRequest(); request_object.setFile(); request_object.setEntityId(); request_object.setFilepath(); request_object.setTimestamp("2018-09-10T05:03:42.363Z"); request_object.setDescription("lorem test3"); request_object.setType("txt"); try { file_service_client.putFile(request_object); } catch (MindsphereException e) { // Exception handling } ``` ### Read a File This section shows two options for reading a file for an asset from the specified path. ```java // Construct the FileServiceClient object as shown above byte[] file_data; try { file_data = file_service_client.getFile(, , ); } catch (MindsphereException e) { // Exception handling } ``` Alternatively, use the [`GetFileRequest`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/iotfileservices/model/GetFileRequest.html) model. ```java // Construct the FileServiceClient object as shown above GetFileRequest request_object = new GetFileRequest(); request_object.setEntityId(); request_object.setFilepath(); byte[] fileData; try { fileData = file_service_client.getFile(request_object); } catch (MindsphereException e) { // Exception handling } ``` ### Delete a File This section shows two options for deleting a file from an asset at the specified path. ```java // Construct the FileServiceClient object as shown above try { file_service_client.deleteFile(, ) } catch (MindsphereException e) { // Exception handling } ``` Alternatively, use the [`DeleteFileRequest`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/iotfileservices/model/DeleteFileRequest.html) model. ```java // Construct the FileServiceClient object as shown above DeleteFileRequest request_object = new DeleteFileRequest(); request_object.setEntityId(); request_object.setFilepath(); try { file_service_client.deleteFile(request_object) } catch (MindsphereException e) { // Exception handling } ``` ### Search a File This section shows two options for searching one or multiple files of an asset in the specified path. ```java // Construct the FileServiceClient object as shown above List file_search_response; try { file_search_response = file_service_client.searchFiles(, , , , , ); } catch (MindsphereException e) { // Exception handling } ``` Alternatively, use the [`GetFileRequest`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/iotfileservices/model/GetFileRequest.html) model. ```java // Construct the FileServiceClient object as shown above SearchFilesRequest request_object = new SearchFilesRequest(); request_object.setEntityId(); List file_search_response; try { file_search_response = file_service_client.searchFiles(request_object); } catch (MindsphereException e) { // Exception handling } ``` ### List Multi Part Uploads This section shows two options for listing multi part uploads. ```java // Construct the FileServiceClient object as shown above List file_list; try { file_list = file_service_client.getFileList(, ); } catch (MindsphereException e) { // Exception handling } ``` Alternatively, use the [`GetFileListRequest`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/iotfileservices/model/GetFileListRequest.html) model. ```java //Create client object "fileServiceClient" as shown above GetFileListRequest request_object = new GetFileListRequest(); request_object.setEntityId(); request_object.setFilepath(); List file_list; try { file_list = file_service_client.getFileList(request_object); } catch (MindsphereException e) { // Exception handling } ``` ### Initiate a New Multi Part Upload Initiate a new multi part file upload for an asset in the specified path. ```java // Construct the FileServiceClient object as shown above // Create the request object PutFileRequest request_object = new PutFileRequest().entityId().filepath(); try { file_service_client.initiateMultiPartUpload(request_object); } catch(MindsphereException e) { // Exception handling } ``` ### Upload Parts for a Multi Part Upload Upload parts of an initiated multi part upload with the provided content in the specified path using the [`PutFileRequest`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/iotfileservices/model/PutFileRequest.html) model. ```java // Construct the PutFileRequest object PutFileRequest request_object = new PutFileRequest() .entityId().filepath() .type().file().part() .timestamp().description(); try { file_service_client.createMultiPartFile(request_object); } catch(MindsphereException e) { // Exception handling } ``` ### Complete a Multi Part Upload Complete a multi part upload using the [`PutFileRequest`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/iotfileservices/model/PutFileRequest.html) model. ```java // Construct the PutFileRequest object as shown above //Create the request object PutFileRequest request_object = new PutFileRequest().entityId().filepath(); try { file_service_client.completeMultiPartUpload(request_object); } catch(MindsphereException e) { // Exception handling } ``` ### Abort a Multi Part Upload Abort a multi part upload using the [`PutFileRequest`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/iotfileservices/model/PutFileRequest.html) model. ```java // Construct the PutFileRequest object as shown above //Create the request object PutFileRequest request_object = new PutFileRequest().entityId().filepath(); try { file_service_client.abortMultiPartUpload(request_object); } catch (MindsphereException e) { // Exception handling } ``` ### Initiate the Update of an Existing Multi Part Upload To update an existing multi part upload, it is required to re-initiate the multi part upload for an asset in the specified path. Note that the `ifMatch` parameter is mandatory for this action. ```java // Construct the PutFileRequest object as shown above //Create the request object PutFileRequest request_object = new PutFileRequest().entityId() .filepath().ifMatch(); try { file_service_client.initiateMultiPartUpload(request_object); } catch(MindsphereException e) { // Exception handling } ``` ### Update an Existing Multi Part Upload Update a part an existing multi part upload with the provided content in the specified path using the [`PutFileRequest`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/iotfileservices/model/PutFileRequest.html) model. Note that the `ifMatch` parameter is mandatory for this action. ```java // Construct the PutFileRequest object as shown above //Create the request object PutFileRequest request_object = new PutFileRequest() .entityId().filepath().ifMatch() .type().file().part() .timestamp().description(); try { file_service_client.updateMultiPartFile(request_object); } catch (MindsphereException e) { // Exception handling } ``` ### Complete the Update of an Existing Multi Part Upload Complete the update of an existing multi part upload using the [`PutFileRequest`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/iotfileservices/model/PutFileRequest.html) model. Note that the `ifMatch` parameter is mandatory for this action. ```java // Construct the PutFileRequest object as shown above //Create the request object PutFileRequest request_object = new PutFileRequest().entityId() .filepath().ifMatch(); try { file_service_client.initiateMultiPartUpload(request_object); } catch(MindsphereException e) { // Exception handling } ``` ### Abort the Update of an Existing Multi Part Upload Abort the update of an existing multi part file upload using the [`PutFileRequest`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/iotfileservices/model/PutFileRequest.html) model. Note that the `ifMatch` parameter is mandatory for this action. ```java // Construct the PutFileRequest object as shown above //Create the request object PutFileRequest request_object = new PutFileRequest().entityId().filepath().ifMatch(); try { file_service_client.abortMultiPartUpload(request_object); } catch (MindsphereException e) { // Exception handling } ``` # IoT Time Series Client for Java ## Introduction The IoT TimeSeries Java client allows you to interact with time series data related to assets. Refer to [IoT Time Series](../../../apis/iot-iottimeseries/api-iottimeseries-overview.html) for more information about the service. Further implementation of the IOT TimeSeries SDK library has been shown in a sample project that you can download and test in local or on Insights Hub application. Please refer to this repository: [industrial-iot-java-sdk-examples](https://github.com/mindsphere/mindsphere-java-sdk-examples) Hint In the IoT context, *assets* are referred to as *entity* and *aspects* as *propertyset*.\ Placeholders in the following samples are indicated by angular brackets `< >`. ## Time Series Operations Client Name: [TimeSeriesClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/timeseries/apiclient/TimeSeriesClient.html) ### Read Time Series Data This section shows two options for achieving the following behavior: - Read time series data for a single aspect of an asset. - Return data for a specified time range. - Return the latest value if no range is provided. - Return time series data for selected fields and limit. ```java // Construct the TimeSeriesClient object TimeSeriesClient timeseries_client = TimeSeriesClient.builder() .mindsphereCredentials() .restClientConfig() .build(); List timeseriesDataList = null; try { String from_time = "2018-01-24T05:03:41.363Z"; String to_time = "2018-01-24T05:04:41.363Z"; Integer limit = 100; String select = null; timeseriesDataList = timeseries_client.getTimeseries(, , from_time, to_time, limit, select); } catch (MindsphereException e) { // Exception handling } ``` Alternatively, use the [`GetTimeseriesRequest`](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-java-v2/javadocs-v2/html/com/siemens/mindsphere/sdk/timeseries/model/GetTimeseriesRequest.html) model. ```java // Construct the TimeSeriesClient object as shown above GetTimeseriesRequest request_object = new GetTimeseriesRequest(); request_object.setEntity(); request_object.setPropertysetname(); request_object.setFrom(); request_object.setTo(); List timeseries_data_list = null; try { timeseries_data_list = timeseries_client.getTimeseries(request_object); } catch (MindsphereException e) { // Exception handling } ``` ### Read Latest Time Series Data This section shows two options for reading the latest value of time series data for a single aspect of an asset. ```java // Construct the TimeSeriesClient object as shown above List timeseries_data = null; try { timeseries_data = timeseries_client.getTimeseries(, , null, null, null, }; let aggregates_data = await aggregates_client.getAggregateTimeseries(request_object); } catch (ex) { // Exception handling } ``` # IoT File Services Client for Node.js ## Introduction The IoT File Services Node.js client allows you to manage files related to assets. Refer to [IoT File Service](../../../apis/iot-iotfile/api-iotfile-overview.html) for more information about the service. Further implementation of the IOT File Service SDK library has been shown in a sample project that you can download and test in local or on Insights Hub application. Please refer to this repository: [industrial-iot-node-sdk-examples](https://github.com/mindsphere/mindsphere-node-sdk-examples) Hint In the IoT context, *assets* are referred to as *entity* and *aspects* as *propertyset*.\ Placeholders in the following samples are indicated by angle brackets `< >`. ## File Operations Client name: [FileServiceClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-node/jsdoc/iotfileservices/index.html#fileserviceclient) ### Create or Update a File Create or update a file with the provided content for an asset in the specified path. ```javascript // Import the mindsphere-sdk-node-core module and require AppCredentials and ClientConfig let AppCredentials = require('mindsphere-sdk-node-core').AppCredentials; let ClientConfig = require('mindsphere-sdk-node-core').ClientConfig; // Import the iotfileservices module and require FileServiceClient const FileServiceClient = require('iotfileservices-sdk').FileServiceClient; // Construct the ClientConfig and AppCredentials objects let config = new ClientConfig(); let credentials = new AppCredentials(); // Construct the FileServiceClient object let file_service_client = new FileServiceClient(config, credentials); try { const request_object = { file: , entityId: , filepath: , ifMatch: , timestamp: , description: , type: }; await file_service_client.putFile(request_object); } catch (ex) { // Exception handling } ``` ### Read a File Read a file for an asset from the specified path. ```javascript // Construct the FileServiceClient object as shown above let file_data; try { const request_object = { entityId: , filepath: , ifNoneMatch: }; file_data = await file_service_client.getFile(request_object); } catch (ex) { // Exception handling } ``` ### Delete a File Delete a file from an asset at the specified path. ```javascript // Construct the FileServiceClient object as shown above try { await file_service_client.deleteFile({ entityId: , filepath: }); } catch (ex) { // Exception handling } ``` ### Search a File Search one or multiple files of an asset in the specified path. ```javascript // Construct the FileServiceClient object as shown above let file_search_response; try { const request_object = { entityId: , offset: , limit: , count: , order: , filter: }; file_search_response = await file_service_client.searchFiles(request_object); } catch (MindsphereException e) { // Exception handling } ``` # IoT Time Series Client for Node.js ## Introduction The IoT Time Series Node.js client allows you to interact with time series data related to assets. Refer to [IoT Time Series](../../../apis/iot-iottimeseries/api-iottimeseries-overview.html) for more information about the service. Further implementation of the IOT TimeSeries SDK library has been shown in a sample project that you can download and test in local or on Insights Hub application. Please refer to this repository: [industrial-iot-node-sdk-examples](https://github.com/mindsphere/mindsphere-node-sdk-examples) Hint In the IoT context, *assets* are referred to as *entity* and *aspects* as *propertyset*.\ Placeholders in the following samples are indicated by angle brackets `< >`. ## Time Series Operations Client Name: [TimeSeriesClient](https://documentation.mindsphere.io/MindSphere/resources/industrial-iot-sdk-node/jsdoc/timeseries/index.html#timeseriesclient) ### Read Time Series Data This operation invokes the IoT Time Series API for the following use cases: - Read time series data for a single aspect of an asset. - Return data for a specified time range. - Return the latest value if no range is provided. - Return time series data for selected fields and limit. ```javascript // Require AppCredentials and ClientConfig from mindsphere-sdk-node-core module let AppCredentials = require('mindsphere-sdk-node-core').AppCredentials; let ClientConfig = require('mindsphere-sdk-node-core').ClientConfig; // Require TimeSeriesClient from timeseries-sdk module const TimeSeriesClient = require('timeseries-sdk').TimeSeriesClient; // Construct the ClientConfig and AppCredentials objects let config = new ClientConfig(); let credentials = new AppCredentials(); // Construct the TimeseriesAggregateClient object let time_series_client = new TimeSeriesClient(config, credentials); let timeseries_data; try { const request_object = { entity: , propertysetname: , from: , to: , limit: , select:
...
` | table | `table_2133` | | **Document** content other than those described above. (For example, paragraphs and lists.) | `

...

` | normal | `polarion_416` | The following Document Parts-related operations are supported: - `GET List of Document Parts` - returns **Document** `homePageContent` as a list of Document Parts elements. - `GET Document Part` - returns specified Document Part by element ID. - `POST Work Item as Document Part` - inserts **Work Item** from the **Document** recycle bin to a right place in the **Document**. - `DELETE Document Parts` - Deletes specified Document Parts from a **Document**. - `POST external Work item as Document Part` - Inserts an external **Work Item** into the intended place in the **Document**. - `POST Overwrite Referenced Work Item Document Parts` - Overwrites multiple **Referenced Work Items** and returns a map of referenced **Work Items** with the new local **Work Item** in the **Document**. ### GET the list of Document Parts You can get a list of all **Document Parts** for a **Document**. - **Example Request:** `GET https://hostname/polarion/rest/v1/projects/{projectId}/spaces/{spaceId}/documents/{documentId}/parts` The **Response Body** below displays the `GET` response for **Headings**, **Work Items**, and **Free text** in a **Document**. - **Response Body:** ```bash { "data": [ { "type": "document_parts", "id": "MyProjectId/MySpaceId/MyDocumentId/heading_MyWorkItemId", "attributes": { "id": "heading_MyWorkItemId", "content": "

", "type": "heading", "headingText": "HeadingText", "level": "0" }, "relationships": { "nextPart": { "data": { "type": "document_parts", "id": "MyProjectId/MySpaceId/MyDocumentId/workitem_MyWorkitemId" } }, "workItem": { "data": { "type": "workitems", "id": "MyProjectId/MyworkitemId" } } }, "links": { "self": "https://hostname/polarion/rest/v1/projects/projectId/spaces/spaceId/documents/documentId/parts/heading_MyWorkItemId" } }, { "type": "document_parts", "id": "MyProjectId/MySpaceId/MyDocumentId/workitem_MyworkitemId", "attributes": { "id": "workitem_MyworkitemId", "content": "
", "type": "workitem", "level": 0, "layout": 0 }, "relationships": { "previousPart": { "data": { "type": "document_parts", "id": "MyProjectId/MySpaceId/MyDocumentId/heading_MyworkitemId" } }, "nextPart": { "data": { "type": "document_parts", "id": "MyProjectId/MySpaceId/MyDocumentId/polarion_freeTextId" } }, "workItem": { "data": { "type": "workitems", "id": "MyProjectId/MyworkitemId" } } }, "links": { "self": "https://hostname/polarion/rest/v1/projects/projectId/spaces/spaceId/documents/documentId/parts/workitem_MyWorkItemId" } }, { "type": "document_parts", "id": "MyProjectId/MySpaceId/MyDocumentId/polarion_freeTextId", "attributes": { "id": "polarion_freeTextId", "content": "

", "type": "normal" }, "relationships": { "previousPart": { "data": { "type": "document_parts", "id": "MyProjectId/MySpaceId/MyDocumentId/workitem_MyWorkItemId" } } }, "links": { "self": "https://hostname/polarion/rest/v1/projects/projectId/spaces/spaceId/documents/documentId/parts/polarion_freeTextId" } } ] } ``` ### GET Document Part - **Example Request:** `GET https://hostname/polarion/rest/v1/projects/{projectId}/spaces/{spaceId}/documents/{documentId}/parts/{partId)` #### GET a Work Item as a Document Part To `GET` a **Work Item** as a **Document Part**. - **Example Request:** `GET https://hostname/polarion/rest/v1/projects/{projectId}/spaces/{spaceId}/documents/{documentId}/parts/{partId)` - **Response Body:** ```bash { "data": { "type": "document_parts", "id": "MyProjectId/MySpaceId/MyDocumentId/workitem_MyWorkItemId", "attributes": { "id": "workitem_MyWorkItemId", "content": "
", "type": "workitem", "level": 0, "layout": 0 }, "relationships": { "previousPart": { "data": { "type": "document_parts", "id": "MyProjectId/MySpaceId/MyDocumentId/workitem_MyWorkItemId" } }, "nextPart": { "data": { "type": "document_parts", "id": "MyProjectId/MySpaceId/MyDocumentId/polarion_template_templateId" } }, "workItem": { "data": { "type": "workitems", "id": "workitem_MyWorkItemId" } } }, "links": { "self": "https://hostname/polarion/rest/v1/projects/projectId/spaces/spaceId/documents/documentId/parts/workitem_MyWorkItemId" } } } ``` #### GET a Heading as a Document Part To `GET` a **Heading** as a **Document Part**. - **Example Request:** `GET https://hostname/polarion/rest/v1/projects/{projectId}/spaces/{spaceId}/documents/{documentId}/parts/{partId)` - **Response Body:** ```bash { "data": { "type": "document_parts", "id": "MyProjectId/MySpaceId/MyDocumentId/heading_MyWorkItemId", "attributes": { "id": "heading_MyWorkitemId", "content": "

", "type": "heading", "headingText": "HeadingText", "level": "0" }, "relationships": { "previousPart": { "data": { "type": "document_parts", "id": "MyProjectId/MySpaceId/MyDocumentId/heading_MyWorkItemId" } }, "nextPart": { "data": { "type": "document_parts", "id": "MyProjectId/MySpaceId/MyDocumentId/heading_MyWorkItemId" } }, "workItem": { "data": { "type": "workitems", "id": "MyProjectId/MyworkitemId" } } }, "links": { "self": "https://hostname/polarion/rest/v1/projects/projectId/spaces/spaceId/documents/documentId/parts/heading_MyWorkItemId" } } } ``` #### GET a Page Break as a Document Part To `GET` a **Page Break** as a **Document Part**. - **Example Request:** `GET https://hostname/polarion/rest/v1/projects/{projectId}/spaces/{spaceId}/documents/{documentId}/parts/{partId)` - **Response Body:** ```bash { "data": { "type": "document_parts", "id": "MyProjectId/MySpaceId/MyDocumentId/pagebreak_pagebreakId", "attributes": { "id": "pagebreak_pagebreakId", "content": "
", "type": "pagebreak", "landscape": true }, "relationships": { "previousPart": { "data": { "type": "document_parts", "id": "MyProjectId/MySpaceId/MyDocumentId/workitem_MyWorkItemId" } }, "nextPart": { "data": { "type": "document_parts", "id": "MyProjectId/MySpaceId/MyDocumentId/workitem_MyWorkItemId" } } }, "links": { "self": "https://hostname/polarion/rest/v1/projects/projectId/spaces/spaceId/documents/documentId/parts/pagebreak_pagebreakId" } } } ``` #### GET Free Text as a Normal Document Part To `GET` **Free Text** as a **Normal** **Document Part**. Note **Normal** refers to other body **Document Parts** like `paragraph`, `pre`, and `list` that are not **Work Items**, **Tables** or other **Document** artifacts. - **Example Request:** `GET https://hostname/polarion/rest/v1/projects/{projectId}/spaces/{spaceId}/documents/{documentId}/parts/{partId)` - **Response Body:** ```bash { "data": { "type": "document_parts", "id": "MyProjectId/MySpaceId/MyDocumentId/polarion_freeTextId", "attributes": { "id": "polarion_freeTextId", "content": "

This document describes couple of requirements related to the configuration (administration) portal.

", "type": "normal" }, "relationships": { "previousPart": { "data": { "type": "document_parts", "id": "MyProjectId/MySpaceId/MyDocumentId/workitem_MyWorkItemId" } }, "nextPart": { "data": { "type": "document_parts", "id": "MyProjectId/MySpaceId/MyDocumentId/workitem_MyWorkItemId" } } }, "links": { "self": "https://hostname/polarion/rest/v1/projects/projectId/spaces/spaceId/documents/documentId/parts/polarion_freeTextId" } } } ``` #### GET a Table of Contents (TOC) as a Document Part To `GET` a **Table of Contents (TOC)** as a **Document Part**. - **Example Request:** `GET https://hostname/polarion/rest/v1/projects/{projectId}/spaces/{spaceId}/documents/{documentId}/parts/{partId)` - **Response Body:** ```bash { "data": { "type": "document_parts", "id": "MyProjectId/MySpaceId/MyDocumentId/toc", "attributes": { "id": "toc", "content": "
", "type": "toc" }, "relationships": { "previousPart": { "data": { "type": "document_parts", "id": "MyProjectId/MySpaceId/MyDocumentId/workitem_MyWorkItemId" } }, "nextPart": { "data": { "type": "document_parts", "id": "MyProjectId/MySpaceId/MyDocumentId/workitem_MyWorkItemId" } } }, "links": { "self": "https://hostname/polarion/rest/v1/projects/projectId/spaces/spaceId/documents/documentId/parts/toc" } } } ``` #### GET a Table of Figures (TOF) as Document Part To `GET` a **Table of Figures (TOF)** as a **Document Part**. - **Example Request:** `GET https://hostname/polarion/rest/v1/projects/{projectId}/spaces/{spaceId}/documents/{documentId}/parts/{partId)` - **Response Body:** ```bash { "data": { "type": "document_parts", "id": "MyProjectId/MySpaceId/MyDocumentId/tof_tofId", "attributes": { "id": "tof_tofId", "content": "
", "type": "tof", "sequence": "table" }, "relationships": { "previousPart": { "data": { "type": "document_parts", "id": "MyProjectId/MySpaceId/MyDocumentId/workitem_MyWorkItemId" } }, "nextPart": { "data": { "type": "document_parts", "id": "MyProjectId/MySpaceId/MyDocumentId/workitem_MyWorkItemId" } } }, "links": { "self": "https://hostname/polarion/rest/v1/projects/projectId/spaces/spaceId/documents/documentId/parts/tof_tofId" } } }{ "data": { "type": "document_parts", "id": "MyProjectId/MySpaceId/MyDocumentId/tof_tofId", "attributes": { "id": "tof_tofId", "content": "
", "type": "tof", "sequence": "table" }, "relationships": { "previousPart": { "data": { "type": "document_parts", "id": "MyProjectId/MySpaceId/MyDocumentId/workitem_MyWorkItemId" } }, "nextPart": { "data": { "type": "document_parts", "id": "MyProjectId/MySpaceId/MyDocumentId/workitem_MyWorkItemId" } } }, "links": { "self": "https://hostname/polarion/rest/v1/projects/projectId/spaces/spaceId/documents/documentId/parts/tof_tofId" } } } ``` #### GET Wiki Blocks as a Document Part To `GET` a **Wiki Block** as a **Document Part**. - **Example Request:** `GET https://hostname/polarion/rest/v1/projects/{projectId}/spaces/{spaceId}/documents/{documentId}/parts/{partId)` - **Response Body:** ```bash { "data": { "type": "document_parts", "id": "elibrary/Specification/Document1/wikiblock_wikiBlockId", "attributes": { "id": "wikiblock_wikiBlockId", "content": "
#documentPanel(true \"approved\")
", "type": "wikiblock", "wikiText": "#documentPanel(true \"approved\")" }, "relationships": { "previousPart": { "data": { "type": "document_parts", "id": "MyProjectId/MySpaceId/MyDocumentId/workitem_MyWorkItemId" } } }, "links": { "self": "https://hostname/polarion/rest/v1/projects/projectId/spaces/spaceId/documents/documentId/parts/wikiblock_wikiBlockId" } } } ``` #### GET Tables as Table Document Parts To `GET` **Tables** as *Tables Document Parts\**. - **Example Request:** `GET https://hostname/polarion/rest/v1/projects/{projectId}/spaces/{spaceId}/documents/{documentId}/parts/{partId)` - **Response Body:** ```bash { "data": { "type": "document_parts", "id": "MyProjectId/MySpaceId/MyDocumentId/table_id", "attributes": { "id": "table_id", "content": "\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
NameTeamTitles
LandoMcLaren1
SebFerrari4
\n", "type": "table" }, "relationships": { "previousPart": { "data": { "type": "document_parts", "id": "MyProjectId/MySpaceId/MyDocumentId/workitem_MyWorkItemId" } }, "nextPart": { "data": { "type": "document_parts", "id": "MyProjectId/MySpaceId/MyDocumentId/workitem_MyWorkItemId" } } }, "links": { "self": "https://hostname/polarion/rest/v1/projects/projectId/spaces/spaceId/documents/documentId/parts/table_id" } } } ``` ### Create Document Parts You can create new **Document Parts** for **Documents**. **Example Request:** `POST https://hostname/polarion/rest/v1/projects/{projectId}/spaces/{spaceId}/documents/{documentId}/parts` ### POST Work Item as Document Part To `POST` a **Work Item** as a Document Part send the following request: - **Example Request:** `POST https://hostname/polarion/rest/v1/projects/{projectId}/spaces/{spaceId}/documents/{documentId}/parts` - **Request Body:** ```bash { "data": [ { "type": "document_parts", "attributes": { "type": "workitem" }, "relationships": { "nextPart": { "data": { "type": "document_parts", "id": "projectId/spaceId/documentId/nextPartId" } }, "previousPart": { "data": { "type": "document_parts", "id": "projectId/spaceId/documentId/previousPartId" } }, "workItem": { "data": { "type": "workitems", "id": "projectId/workItemId" } } } } ] } ``` Note Only one of the `previousPart`/`nextPart` fields can be specified. If `previousPart`/`nextPart` is not specified, the **Work Item** is inserted at the end of the Document content. The specified **Work Item** already has to exist and be present in the **Document's** recycle bin. See [Create a Work Item in a Document](advanced-concepts.html#create-a-work-item-in-a-document) for more information.. - **Response Body:** ```bash { "data": [ { "type": "document_parts", "id": "projectId/spaceId/documentId/partId", "links": { "self": "https://hostname/polarion/rest/v1/projects/projectId/spaces/spaceId/documents/documentId/parts/documentPartId" } } ] } ``` #### POST a Heading as a Document Part To `POST` a **Heading** as a **Document Part** send the following request: - **Example Request:** `POST https://hostname/polarion/rest/v1/projects/{projectId}/spaces/{spaceId}/documents/{documentId}/parts` - **Request Body:** ```bash { "data": [ { "type": "document_parts", "attributes": { "headingText": "HeadingText", "level": 0, "type": "heading" }, "relationships": { "nextPart": { "data": { "type": "document_parts", "id": "projectId/spaceId/documentId/nextPartId" } }, "previousPart": { "data": { "type": "document_parts", "id": "projectId/spaceId/documentId/previousPartId" } } } } ] } ``` Note Only one of the `previousPart`/`nextPart` fields can be specified. If one of them is not specified, the *Heading* is inserted at the end of the *Document* content. - **Response Body:** ```bash { "data": [ { "type": "document_parts", "id": "MyProjectId/MySpaceId/MyDocumentId/workitem_MyWorkItemId", "links": { "self": "https://hostname/polarion/rest/v1/projects/projectId/spaces/spaceId/documents/documentId/parts/docu…" } } ] } ``` #### POST a Page Break as a Document Part **Page Breaks** as a **Document Parts**. To `POST` a **Page Break** as a **Document Part** send the following request: - **Example Request:** `POST https://hostname/polarion/rest/v1/projects/{projectId}/spaces/{spaceId}/documents/{documentId}/parts` - **Request Body:** ```bash { "data": [ { "type": "document_parts", "attributes": { "type": "pagebreak", "landscape": false/true }, "relationships": { "nextPart": { "data": { "type": "document_parts", "id": "MyProjectId/MySpaceId/MyDocumentId/workitem_MyWorkItemId" } }, "previousPart": { "data": { "type": "document_parts", "id": "MyProjectId/MySpaceId/MyDocumentId/workitem_MyWorkItemId" } } } } ] } ``` Note **Remember** - The `previousPart` and `nextPart` parameters specify the position of the **Page Break** in the **Document**, either by pointing to the part it should follow or precede. - Only one of the previousPart / nextPart parameters can be specified, and it must be the **ID** of an existing part of the target **Document**. Specifying neither `previousPart` nor `nextPart` results in adding the **Page Break** at the end of the **Document**. - **Response Body:** ```bash { "data": [ { "type": "document_parts", "id": "MyProjectId/MySpaceId/MyDocumentId/workitem_MyWorkItemId", "links": { "self": "https://hostname/polarion/rest/v1/projects/projectId/spaces/spaceId/documents/documentId/parts/docu…" } } ] } ``` #### POST Free Text as a Normal Document Part **Free Text** (paragraphs, lists, and preformatted text) can be added as a **Normal Document Part**. To `POST` **Free Text** as a **Normal Document Part** send the following request: - **Example Request:** `POST https://hostname/polarion/rest/v1/projects/{projectId}/spaces/{spaceId}/documents/{documentId}/parts` - **Request Body:** ```bash { "data": [ { "type": "document_parts", "attributes": { "type": "normal", "content": "

Free Text to add

" }, "relationships": { "nextPart": { "data": { "type": "document_parts", "id": "projectId/spaceId/documentId/nextPartId" } }, "previousPart": { "data": { "type": "document_parts", "id": "projectId/spaceId/documentId/previousPartId" } } } } ] } ``` - **Response Body:** ```bash { "data": [ { "type": "document_parts", "id": "projectId/spaceId/documentId/partId", "links": { "self": "https://hostname/polarion/rest/v1/projects/projectId/spaces/spaceId/documents/documentId/parts/docu…" } } ] } ``` Note The content field only allows the following four tags as the outermost tags: `

` , `

    ` ,`
      ` and `
      `.

#### POST Table of Contents (TOC) as a Document Part

**Table of Contents** (TOCs) can be added as a **Document Part**.

To `POST` a **TOC** as a **Document Part** send the following request:

- **Example Request:**

`POST https://hostname/polarion/rest/v1/projects/{projectId}/spaces/{spaceId}/documents/{documentId}/parts`

- **Request Body:**

```bash
{
  "data": [
    {
      "type": "document_parts",
      "attributes": {
        "type": "toc"
      },
      "relationships": {
        "nextPart": {
          "data": {
            "type": "document_parts",
            "id": "MyProjectId/MySpaceId/MyDocumentId/workitem_MyWorkItemId"
          }
        }
      }
    }
  ]
}

```

Note

**Remember**

- The `previousPart` and `nextPart` parameters specify the position of the **Table of Contents** in the **Document**, either by pointing to the part it should follow or precede.

- Only one of the `previousPart` / `nextPart` parameters can be specified, and it must be the **ID** of an existing part of the target **Document**. Specifying neither `previousPart` nor `nextPart` results in adding the **TOC** at the end of the **Document**.

- Only one **TOC** can be added to a **Document**.

- **Response Body:**

```bash
{
  "data": [
    {
      "type": "document_parts",
      "id": "projectId/spaceId/documentId/partId",
      "links": {
        "self": "https://hostname/polarion/rest/v1/projects/projectId/spaces/spaceId/documents/documentId/parts/docu…"
      }
    }
  ]
}

```

#### POST Table of Figures (TOF) as a Document Part

A **Table of Contents** (TOFs) can be added as a **Document Part**.

To `POST` a **TOF** as a **Document Part** send the following request:

- **Example Request:**

`POST https://hostname/polarion/rest/v1/projects/{projectId}/spaces/{spaceId}/documents/{documentId}/parts`

- **Request Body:**

```bash
{
  "data": [
    {
      "type": "document_parts",
      "attributes": {
        "type": "tof",
        "sequence": "Table"
      },
      "relationships": {
        "nextPart": {
          "data": {
            "type": "document_parts",
            "id": "MyProjectId/MySpaceId/MyDocumentId/workitem_MyWorkItemId"
          }
        }
      }
    }
  ]
}

```

Note

**Remember**

- The `previousPart` and `nextPart` parameters specify the position of the **Table of Figures** in the **Document**, either by pointing to the part it should follow or precede.

- Only one of the `previousPart` / `nextPart` parameters can be specified, and it must be the **ID** of an existing part of the target **Document**. Specifying neither `previousPart` nor `nextPart` results in adding the **TOF** at the end of the **Document**.

- The **Document** must include a sequence that is created and attached as a label to support **TOF** generation.

- **Response Body:**

```bash
{
  "data": [
    {
      "type": "document_parts",
      "id": "projectId/spaceId/documentId/partId",
      "links": {
        "self": "https://hostname/polarion/rest/v1/projects/projectId/spaces/spaceId/documents/documentId/parts/docu…"
      }
    }
  ]
}

```

#### POST Wiki Blocks as a Document Part

**Wiki Blocks** can be added as **Document Parts**.

To `POST` **Wiki Blocks** as a **Document Part** send the following request:

- **Example Request:**

`POST https://hostname/polarion/rest/v1/projects/{projectId}/spaces/{spaceId}/documents/{documentId}/parts`

- **Request Body:**

```bash
{
  "data": [
    {
      "type": "document_parts",
      "attributes": {
        "type": "wikiblock",
        "wikiText": "#documentPanel(true \"approved\")"
      },
      "relationships": {
        "nextPart": {
          "data": {
            "type": "document_parts",
            "id": "MyProjectId/MySpaceId/MyDocumentId/workitem_MyWorkItemId"
          }
        },
        "previousPart": {
          "data": {
            "type": "document_parts",
            "id": "MyProjectId/MySpaceId/MyDocumentId/workitem_MyWorkItemId"
          }
        }
      }
    }
  ]
}

```

Note

**Remember**

- The `previousPart` and `nextPart` parameters specify the position of the **Wiki Blocks** in the **Document**, either by pointing to the part it should follow or precede.

- Only one of the `previousPart` / `nextPart` parameters can be specified, and it must be the **ID** of an existing part of the target **Document**. Specifying neither `previousPart` nor `nextPart` results in adding the **Wiki Blocks** at the end of the **Document**.

- The **Document** must include a sequence that is created and attached as a label to support **Wiki Blocks** generation.

- **Response Body:**

```bash
{
  "data": [
    {
      "type": "document_parts",
      "id": "projectId/spaceId/documentId/partId",
      "links": {
        "self": "https://hostname/polarion/rest/v1/projects/projectId/spaces/spaceId/documents/documentId/parts/documentPartId"
      }
    }
  ]
}

```

#### POST Tables as a Table Document Part

**Tables** can be added as a Table part into a **Document**.

To `POST` **Tables** as a **Table Document Part** send the following request.

- **Example Request:**

`POST https://hostname/polarion/rest/v1/projects/{projectId}/spaces/{spaceId}/documents/{documentId}/parts`

- **Request Body:**

```bash
{
  "data": [
    {
      "type": "document_parts",
      "attributes": {
        "type": "table",
        "content": "
      NameTeamTitles
      LandoMcLaren1
      SebFerrari4
      "
      },
      "relationships": {
        "nextPart": {
          "data": {
            "type": "document_parts",
            "id": "projectId/spaceId/documentId/nextPartId"
          }
        },
        "previousPart": {
          "data": {
            "type": "document_parts",
            "id": "projectId/spaceId/documentId/previousPartId"
          }
        }
      }
    }
  ]
}

```

Note

You can only specify one of the `previousPart` / `nextPart` fields. If `previousPart` / `nextPart` is not specified, the **Table** is inserted at the end of the **Document** content.

- **Response:**

```bash
{
  "data": [
    {
      "type": "document_parts",
      "id": "projectId/spaceId/documentId/partId",
      "links": {
        "self": "https://hostname/polarion/rest/v1/projects/projectId/spaces/spaceId/documents/documentId/parts/docu…"
      }
    }
  ]
}

```

### Additional information

To generate multiple **Document Parts** sequentially after a specific **Document Part**, use its **ID** as the `previousPart` or `nextPart` in the relationships of each **Document Part** in the payload.

-**Example**

There are three existing **Normal** parts in a **Document**.

To create three new **Document Parts** between the `polarion_2` and `Polarion_3` parts, you would do the following:

```bash
{
  "data": [
    {
      "type": "document_parts",
      "attributes": {
        "content": "
      New1
      ",
        "type": "normal"
      },
      "relationships": {
        "nextPart": {
          "data": {
            "type": "document_parts",
            "id": "A/_default/doc2/polarion_3"
          }
        }
      }
    },
    {
      "type": "document_parts",
      "attributes": {
        "content": "
      New2
      ",
        "type": "normal"
      },
      "relationships": {
        "nextPart": {
          "data": {
            "type": "document_parts",
            "id": "A/_default/doc2/polarion_3"
          }
        }
      }
    },
    {
      "type": "document_parts",
      "attributes": {
        "content": "
      New3
      ",
        "type": "normal"
      },
      "relationships": {
        "nextPart": {
          "data": {
            "type": "document_parts",
            "id": "A/_default/doc2/polarion_3"
          }
        }
      }
    }
  ]
}

```

#### Before

#### After

Note

If you use `previousPart` as a reference, then you must provide the parts in reverse order in the payload to get the same sequence in the Polarion UI.

#### DELETE Document Parts from a Document

This endpoint allows you to delete multiple **Document Parts** from a **Document**.

- **Example Request:**

`DELETE https://hostname/polarion/rest/v1/projects/{projectId}/spaces/{spaceId}/documents/{documentName}/parts`

- **Request Body:**

```bash
{
  "data": [
    {
      "type": "document_parts",
      "id": "ss/_default/doc/workitem_SS-533"
    }
  ]
}

```

- **Response:**

```bash
204 - No Content

```

#### Move a Work Item Part in a Document

This endpoint allows you to modify a **Document's** structure by moving the selected **Work Item Document Part**.

Note

**The following applies to all endpoints that Move a Work Item to a Document Part.**

- This endpoint moves the target **Work Item Document Part**, along with its children, under the specified parent.

Its position in the parent **Work Item** is determined by the value of the before or after fields.

```text
- If the target **Work Item Document Part** is already a child of the specified parent, and no before or after position is specified, it is moved to the end of the child list.

- In an empty request (where no parent, before, or after position is supplied), the target **Work Item** is placed at the end of the **Document**.

- If the target **Work Item** is referenced in the **Document**, then its position in the **Document** is changed without any change to its parent relationship.

- If the parent field is not provided, the target **Work Item Document Part** is moved under the root node of the **Document**.

```

- **Example Request:**

`POST /projects/{projectId}/spaces/{spaceId}/documents/{documentName}/parts/{partId}/actions/move`

- **Request Body:**

```bash
{
  "before": "MyProjectId/MySpaceId/MyDocumentId/workitem_MyBeforeWorkItemId",
  "after": "MyProjectId/MySpaceId/MyDocumentId/workitem_MyAfterWorkItemId",
  "parent": "MyProjectId/MySpaceId/MyDocumentId/workitem_MyParentWorkItemId"
}

```

- **Response:**

```bash
204 - No Content

```

##### Move Up

**Move Up** moves a **Work Item Document Part**, along with its children, higher in the order under the specified parent. Its new position within the parent is determined by using the `before` or `after` fields.

- **Example:** Move `Item13` up so that it appears immediately before `Item11` under the parent `Item1`.

```bash
*Heading H1
  * Item 1 (level 0)
    |--> Item 11 (level 1)
    |--> Item 12 (level 1)
    |--> Item 13 (level 1)

```

- **Request:**

`POST /projects/{projectId}/spaces/{spaceId}/documents/{documentName}/parts/{workitem_Item13}/actions/move`

- **Request Body:**

```bash
{
  "before": "MyProjectId/MySpaceId/MyDocumentId/workitem_Item11",
  "parent": "MyProjectId/MySpaceId/MyDocumentId/workitem_Item1"
}

```

- **Response Body:**

```bash
204 - No Content

```

After a successful modification (a `204` response), the updated structure is as follows:

```bash
*Heading H1
  * Item 1 (level 0)
    |--> Item 13 (level 1)
    |--> Item 11 (level 1)
    |--> Item 12 (level 1)

```

##### Move Down

**Move Down** moves a **Work Item** Document Part, along with its children, lower in the order under the specified parent. Its new position within the parent is determined by using the `before` or `after` fields.

You can **Move Down** a selected **Work Item** Document Part just like you **Move** them **Up**.

- **Example:** Move `Item13` down so that it appears immediately before `Item12` under the parent `Item1`.

```bash
*Heading H1
  * Item 1 (level 0)
    |--> Item 13 (level 1)
    |--> Item 11 (level 1)
    |--> Item 12 (level 1)

```

- **Request:**

`POST /projects/{projectId}/spaces/{spaceId}/documents/{documentName}/parts/{workitem_Item13}/actions/move`

- **Request Body:**

```bash
{
  "after": "MyProjectId/MySpaceId/MyDocumentId/workitem_Item12",
  "parent": "MyProjectId/MySpaceId/MyDocumentId/workitem_Item1"
}

```

- **Response Body:**

```bash
204 - No Content

```

After a successful modification (a `204` response), the updated structure is as follows:

```bash
*Heading H1
  * Item 1 (level 0)
    |--> Item 11 (level 1)
    |--> Item 12 (level 1)
    |--> Item 13 (level 1)

```

##### Promoting a Work Item in a Document Structure

Promoting a **Work Item Document Part** changes its parent, which changes its level in the hierarchy.

(When promoted, it's moved up in the hierarchy.)

- **Example:** `Item111` (currently at Level `2`) is moved so that it appears immediately after `Item3` (Level `0`) under the parent heading `H1`. (The item gets promoted from Level `2` to Level `0`.)

```bash
*Heading H1
  * Item 1 (level 0)
    |--> Item 11 (level 1)
          |--> Item 111 (level 2)
  * Item 2 (level 0)
  * Item 3 (level 0)
    |--> Item 31 (level 1)

```

- **Request:**

`POST /projects/{projectId}/spaces/{spaceId}/documents/{documentName}/parts/{workitem_Item111}/actions/move`

- **Request Body:**

```bash
{
  "after": "MyProjectId/MySpaceId/MyDocumentId/workitem_Item3",
  "parent": "MyProjectId/MySpaceId/MyDocumentId/Heading_H1"
}

```

- **Response Body:**

```bash
204 - No Content

```

After a successful modification (a `204` response), the updated structure is as follows:

```bash
*Heading H1
  * Item 1 (level 0)
    |--> Item 11 (level 1)
  * Item 2 (level 0)
  * Item 3 (level 0)
    |--> Item 31 (level 1)
  *  Item 111 (level 0)

```

##### Demoting a Work Item in a Document Structure

Demoting a **Work Item Document Part** changes its parent, which changes its level in the hierarchy.

(When demoted, it's moved down in the hierarchy.)

- **Example:** `Item31` (currently at Level `1`) is moved so that it appears immediately before `Item111` (Level `2`) under the parent `Item1`. (The item gets demoted from Level `1` to Level `2`.)

```bash
*Heading H1
  * Item 1 (level 0)
    |--> Item 11 (level 1)
    |--> Item 111 (level 2)
  * Item 2 (level 0)
  * Item 3 (level 0)
    |--> Item 31 (level 1)

```

The corresponding request will be:

- **Request:**

`POST /projects/{projectId}/spaces/{spaceId}/documents/{documentName}/parts/{workitem_Item31}/actions/move`

- **Request Body:**

```bash
{
  "before": "MyProjectId/MySpaceId/MyDocumentId/workitem_Item111",
  "parent": "MyProjectId/MySpaceId/MyDocumentId/workitem_Item1"
}

```

- **Response Body:**

```bash
204 - No Content

```

After a successful modification (a `204` response), the updated structure is as follows:

```bash
*Heading H1
  * Item 1 (level 0)
    |--> Item 11 (level 1)
    |--> Item 31 (level 2)
    |--> Item 111 (level 2)
  * Item 2 (level 0)
  * Item 3 (level 0)

```

#### Overwrite Referenced Work Item Document Parts

Send a `POST` request with **Referenced Work Items** (as **Document Parts**) to create localized copies inside the **Document**.

The endpoint overwrites the existing references and replaces them with newly created local **Work Items** within the **Document**.

- **Example Request:**

`POST https://hostname/polarion/rest/v1/projects/{projectId}/spaces/{spaceId}/documents/{documentName}/parts/actions/overwrite`

- **Request Body:**

```bash
{
  "partIds": [
    "MyProjectId/MySpaceId/MyDocumentId/workitem_workItem1",
    "MyProjectId/MySpaceId/MyDocumentId/workitem_workItem2"
  ]
}

```

- **Response Body:**

```bash
{
  "data": {
    "overwrittenParts": [
      {
        "oldPartId": "MyProjectId/MySpaceId/MyDocumentId/workitem_workItem1",
        "newPartId": "MyProjectId/MySpaceId/MyDocumentId/workitem_workItem3"
      },
      {
        "oldPartId": "MyProjectId/MySpaceId/MyDocumentId/workitem_workItem2",
        "newPartId": "MyProjectId/MySpaceId/MyDocumentId/workitem_workItem4"
      }
    ]
  }
}{
  "data": {
    "overwrittenParts": [
      {
        "oldPartId": "MyProjectId/MySpaceId/MyDocumentId/workitem_workItem1",
        "newPartId": "MyProjectId/MySpaceId/MyDocumentId/workitem_workItem3"
      },
      {
        "oldPartId": "MyProjectId/MySpaceId/MyDocumentId/workitem_workItem2",
        "newPartId": "MyProjectId/MySpaceId/MyDocumentId/workitem_workItem4"
      }
    ]
  }
}

```

### POST external Work Item as Document Part

External **Work Items**, items not part of the **Document** and have no module relationship, can be added as referenced **Work Items** in the **Document**.

To `POST` an external **Work Item** as a **Document Part** send the following request:

- **Example Request:**

`POST https://hostname/polarion/rest/v1/projects/{projectId}/spaces/{spaceId}/documents/{documentId}/parts`

- **Request Body:**

```bash
{ 
  "data": [ 
   { 
     "type": "document_parts", 
     "attributes": {
      "type": "workitem"
     },  
     "relationships": {
       "nextPart": {
        "data": {  
         "type": "document_parts",  
         "id": "projectId/spaceId/documentId/nextPartId"
        }
      },  
      "previousPart": {  
        "data": {  
         "type": "document_parts",  
         "id": "projectId/spaceId/documentId/previousPartId"
        }  
      },  
      "workItem": {  
        "data": {  
         "type": "workitems",  
         "id": "projectId/workItemId"
        }  
       }  
      }  
     }  
    ] 
   }

```

- **Response Body:**

```bash
{  
   "data": [
     {  
       "type": "document_parts",  
       "id": "projectId/spaceId/documentId/partId",
      "links": {  
        "self": 
"https://hostname/polarion/rest/v1/projects/projectId/spaces/spaceId/documents/documentId/parts/documentPartId"  
        }  
       }
     ] 
    } 

```

### POST Document Part with level

The `POST` **Document Parts** endpoint provides the possibility to specify the indent level for the new **Document Part**.

The default level is `0`.

You can change the level of an already existing **Document Part** by PATCHing the **Document** `homePageContent`.

#### Example Request

`POST https://hostname/polarion/rest/v1/projects/{projectId}/spaces/{spaceId}/documents/{documentId}/parts`

- **Request Body:**

```bash
{
  "data": [
    {
      "type": "document_parts",
      "level": 1,
      "attributes": {
        "type": "workitem"
      },
      "relationships": {

        "previousPart": {
          "data": {
            "type": "document_parts",
            "id": "projectId/spaceId/documentId/previousPartId"
          }
        },

        "workItem": {
          "data": {
            "type": "workitems",
            "id": "projectId/workItemId"
          }
        }
      }
    }
  ]
}

```

- **Response Body:**

```bash
{
  "data": [
    {
      "type": "document_parts",
      "id": "projectId/spaceId/documentId/partId",
      "links": {
        "self": "https://hostname/polarion/rest/v1/projects/projectId/spaces/spaceId/documents/documentId/parts/documentPartId"
      }
    }
  ]
}

```

### POST Document Part with layout

The `POST` **Document Parts** endpoint allows you to specify a layout for a new **Document Part** by providing its zero-based index from the list of available layouts in the **Document**.

```bash
{
  "outlineNumbering": {
    "prefix": "ABC"
  },
  "renderingLayouts": [
    {
      "type": "task",
      "label": "My label",
      "layouter": "paragraph",
      "properties": [
        {
          "key": "fieldsAtStart",
          "value": "id"
        }
      ]
    }
  ]
}

```

Note

You can retrieve the **Document's** layout with the `GET` document endpoint:

`GET /projects/{projectId}/spaces/{spaceId}/documents/{documentName}`

By default, the first layout for the corresponding **Work Item** type from the layout list is applied. If no layout exists for that **Work Item** type, the first layout from the layout list is used instead.

You can change the layout of an already existing Document Part by `PATCHing` the **Document** `homePageContent`.

- **Example Request:**

`POST https://hostname/polarion/rest/v1/projects/{projectId}/spaces/{spaceId}/documents/{documentId}/parts`

- **Request Body:**

```bash
{
  "data": [
    {
      "type": "document_parts",
      "layout": 1,
      "attributes": {
        "type": "workitem"
      },
      "relationships": {
        "previousPart": {
          "data": {
            "type": "document_parts",
            "id": "projectId/spaceId/documentId/previousPartId"
          }
        },
        "workItem": {
          "data": {
            "type": "workitems",
            "id": "projectId/workItemId"
          }
        }
      }
    }
  ]
}

```

- **Response Body:**

```bash
{
  "data": [
    {
      "type": "document_parts",
      "id": "projectId/spaceId/documentId/partId",
      "links": {
        "self": "https://hostname/polarion/rest/v1/projects/projectId/spaces/spaceId/documents/documentId/parts/documentPartId"
      }
    }
  ]
}

```

## Document comments

You can fetch, update (resolve or reopen), and create **Document** comments via **Document** Comments endpoints. You can create `Replies` comments by a `POST` endpoint specifying the `parentComment` relationship. Without that relationship a top-level comment is created. A new top-level comment created by the REST API appears as `Unreferenced`, meaning that it does not appear in the Document's content. To add it to the content, also update the `homePageContent` field of the **Document** with the `PATCH` **Document** endpoint.

For example, the comment is created first:

- **Example Request:**

```bash
POST https://hostname/polarion/rest/v1/projects/{projectId}/spaces/{spaceId}/documents/{documentId}/comments

```

- **Request Body**

```bash
{
  "data": [
    {
      "type": "document_comments",
      "attributes": {
        "text": {
          "type": "text/html",
          "value": "My text value"
        }
      },
      "relationships": {
        "author": {
          "data": {
            "type": "users",
            "id": "userId"
          }
        }
      }
    }
  ]
}

```

And then the **Document's** content is updated by adding a `` element to the desired position.

Note

The whole value of `homePageContent` needs to be sent in the request body, you cannot send the updated or inserted part only.

- **Example Request:**

```bash
PATCH https://hostname/polarion/rest/v1/projects/{projectId}/spaces/{spaceId}/documents/{documentId}

```

- **Request Body**

```bash
{
  "data":{
     "type":"documents",
     "id":"projectId/spaceId/documentId",  
     "attributes":{ 
        "homePageContent":{  
           "type":"text/html",  
           "value":"......" 
        } 
      } 
   } 
}

```

Note

If the new top-level comment is to be placed in some of the **Document's Work Items**, the description of the corresponding **Work Item** resource has to be updated with the same `span` element instead of **Document's** content.

## Branch Documents

### Branch a single Document

To branch a **Document**, use the **Document's** resource URL and send a POST request with a special request body like the example below. You can include the `revision` URL parameter to create a from a specific revision. Without the `revision` parameter, it will branch from the HEAD. You can also include `query` in the request body field to specify the **Work Items** to branch.

- **Example Request:**

```bash
POST https://myhost/polarion/rest/v1/projects/myProject/spaces/mySpace/documents/MyDocument/actions/branch?revision=2345

```

- **Request Body**

```bash
{
    "targetProjectId": "myOtherProject",
    "targetSpaceId": "myOtherSpace",
    "targetDocumentName": "MyDocumentBranched",
    "copyWorkflowStatusAndSignatures": false,
    "query": "status:draft"
}}

```

- **Response Body**

```bash
{
    "data": {
        "type": "documents",
        "id": "myOtherProject/myOtherSpace/MyDocumentBranched",
        "attributes": {...},
        "relationships": {...},
        "links": {
            "self": "https://myhost/polarion/rest/v1/projects/myOtherProject/spaces/myOtherSpace/documents/MyDocumentBranched"
        }
    }
}

```

### Branch several Documents at once

To branch several Documents at once, use the `/all/documents/actions/branch` URL and send a `POST` request with a special Request Body like in the example below. This action is executed asynchronously, so in the response you receive the ID of a job that was started to perform the task. You can then use another `/jobs/{jobId}` endpoint to get information about the job's status. If the job finished successfully, the job response contains a `documents` relationship to all the branched **Documents**.

- **Example Request:**

```bash
POST https://myhost/polarion/rest/v1/all/documents/actions/branch

```

- **Request Body**

```bash
{
  "documentConfigurations": [
    {
      "sourceDocument": "myProject/mySpace/myDocument1",
      "targetDocumentName": "myDocument1_branched"
    },
    {
      "sourceDocument": "myProject/mySpace/myDocument2",
      "targetDocumentName": "myDocument2_branched"
    }
  ]
}

```

- **Response Body**

```bash
{
  "data": {
    "type": "jobs",
    "id": "6fb118b2-927aec93-13c0ef19-91ebc29f",
    "attributes": {
      "jobId": "6fb118b2-927aec93-13c0ef19-91ebc29f",
      "name": "Branch Documents",
      "state": "RUNNING"     },
    "links": {...}
  }
}

```

- **Example Request:**

```bash
GET https://myhost/polarion/rest/v1/jobs/6fb118b2-927aec93-13c0ef19-91ebc29f?fields[jobs]=@all

```

- **Response Body**

```bash
{
  "links": {...},
  "data": {
    "type": "jobs",
    "id": "6fb118b2-927aec93-13c0ef19-91ebc29f",
    "attributes": {
      "jobId": "6fb118b2-927aec93-13c0ef19-91ebc29f",
      "name": "Branch Documents",
      "state": "FINISHED",
      "status": {
        "type": "OK"
      }
    },
    "relationships": {
      "documents": {
        "data": [
          {
            "type": "documents",
            "id": "myProject/mySpace/myDocument1_branched"
          },
          {
            "type": "documents",
            "id": "myProject/mySpace/myDocument2_branched"
          }
        ]
      }
    },
    "links": {...}
  }
}

```

### Copy a Document

To copy a **Document**, use the **Document's** resource URL and send a `POST` request with a special request body like the example below. You can include the revision URL parameter to create a copy from a specific revision. Without the revision parameter, it will copy the HEAD revision.

- **Example Request:**

```bash
POST https://myhost/polarion/rest/v1/projects/myProject/spaces/mySpace/documents/MyDocument/actions/copy?revision=2345

```

- **Request Body**

```bash
{
    "targetProjectId": "myOtherProject",
    "targetSpaceId": "myOtherSpace",
    "targetDocumentName": "MyDocumentCopied",
    "removeOutgoingLinks": true,
    "linkOriginalItemsWithRole": "duplicates"
}

```

- **Response Body**

```bash
{
    "data": {
        "type": "documents",
        "id": "myOtherProject/myOtherSpace/MyDocumentCopied",
        "attributes": {...},
        "relationships": {...},
        "links": {
            "self": "https://myhost/polarion/rest/v1/projects/myOtherProject/spaces/myOtherSpace/documents/MyDocumentCopied"
        }
    }
}

```

### Merge Documents

To merge **Work Items** between **Branched** and **Master Documents**, use the **Branch Document's** resource URL and send a `POST` request with a special request body like the example below.

To merge **Work Items** from **Branch to Master**, call the `/mergeToMaster` action. To merge **Work Items** from **Master to Branch**, call the `/mergeFromMaster` action.

This action is executed asynchronously, so you receive the **ID** of a job that was started to perform the task in the response. You can then use the `/jobs/{jobId}` endpoint to get information about the job's status.

If the job finishes successfully, the job response contains a document relationship to the **Target Document** of the merge.

If the merge finished with some changes in the **Target Document**, the relationship in the response contains the revision number created by the merge changes.

### Example Request

`POST` `https://myhost/polarion/rest/v1/projects/myProject/spaces/mySpace/documents/MyDocument/actions/mergeToMaster`

`https://myhost/polarion/rest/v1/projects/myProject/spaces/mySpace/documents/MyDocument/actions/mergeFromMaster`

### Request Body

```bash
{

  "createBaseline": true, 
  "userFilter": "status:open"  

}

```

### Response Body

```bash
{ 
  "data": {
    "type": "jobs",
    "id": "6fb118b2-927aec93-13c0ef19-91ebc29f",  
    "attributes": {  
      "jobId": "6fb118b2-927aec93-13c0ef19-91ebc29f",  
      "name": "Automatic Merge Documents",  
      "state": "RUNNING"
     },  
     "links": {...}
  } 
}

```

### Example Request

`GET https://myhost/polarion/rest/v1/jobs/6fb118b2-927aec93-13c0ef19-91ebc29f?fields[jobs]=@all`

### Responses

**A Successful job: A new revision is created for the Target Document as a result of the Automatic Merge Job:**

```bash
{  
  "data": {
    "type": "jobs",  
    "id": "6fb118b2-927aec93-13c0ef19-91ebc29f",  
    "attributes": {
    "jobId": "6fb118b2-927aec93-13c0ef19-91ebc29f",
    "name": "Automatic Merge Documents",
    "state": "FINISHED",
    "status": { 
      "type": "OK"
    }  
  },  
  "relationships": {  
    "document": {
      "data": {  
      "type": "documents",  
        "id": "MyProjectId/MySpaceId/TargetDocumentId",
        "revision": "example"
     }  
    }  
   },  
   "links": {
     "self": "https://localhost:8888/polarion/rest/v1/jobs/6fb118b2-927aec93-13c0ef19-91ebc29f",
     "log": "https://localhost:8888/polarion/job-report?jobId=MyJobId"
     }
   } 
}

```

**A Successful job: No changes in the Target Document as a result of the Automatic Merge Job:**

```bash
{  
    "data": {
      "type": "jobs",
      "id": "6fb118b2-927aec93-13c0ef19-91ebc29f",
      "attributes": {
        "jobId": "6fb118b2-927aec93-13c0ef19-91ebc29f",
        "name": "Automatic Merge Documents",
        "state": "FINISHED",
        "status": {
        "type": "OK"}
        },
        "relationships": {
          "document": {
          "data": {
          "type": "documents",
          "id": "MyProjectId/MySpaceId/TargetDocumentId",
        }
        }
      },
      "links": {
        "self": "https://localhost:8888/polarion/rest/v1/jobs/MyJobId",
        "log": "https://localhost:8888/polarion/job-report?jobId=6fb118b2-927aec93-13c0ef19-91ebc29f"
      }
     } 
}

```

## Enumerations

Enumeration resources are identified by `project ID`, enumeration context (**Document**, **Plans**, **Testing**, or no context), enumeration name, and target type. Enumeration context and enumeration name together (separated by "/", if there is a context) form the enumeration ID known from Java API.

See the table below for examples.

| Enumeration ID (Java API) | Enumeration Context | Enumeration Name | Target Type | Note                                                                                      |
| ------------------------- | ------------------- | ---------------- | ----------- | ----------------------------------------------------------------------------------------- |
| severity                  |                     | severity         | testcase    | A testcase-specific enumeration of severities.                                            |
| documents/document-status | documents           | document-status  |             | A general (not specific to any **Document** type) configuration of **Document** statuses. |
| @document                 |                     | @document        |             | An object enumeration of **Documents**.                                                   |

Note

An empty enumeration context or empty target type is represented in the REST API request URLs by the `~ path` parameter.

(Because the `path` parameters cannot be empty.)

In the following sections we describe operations related to enumerations:

- CRUD operations for the enumerations
- [GET all enumerations in the Project context](advanced-concepts.html#get-all-enumerations-in-the-project-context)
- [GET all enumerations in the Global context](advanced-concepts.html#get-all-enumerations-in-the-global-context)
- [Get options related to enumeration fields](advanced-concepts.html#get-options-related-to-enumeration-fields)
- [Read available enumeration icons](advanced-concepts.html#read-available-enumeration-icons)
- [Create enumeration icons](advanced-concepts.html#create-enumeration-icons)
- [Custom Enumeration Object Factories](advanced-concepts.html#custom-enumeration-object-factories)

### CRUD operations for the enumerations

CRUD operations are supported for enumerations, however, not all enumerations support all CRUD operations. Some enumerations are not configurable at all - there are constant enumerations (for example `approval-status`) or object enumerations (for example `@document`). Such configurations can only be read.

Options that comprise the enumeration are stored in an `options` attribute (so options themselves are not resources), see the example of a GET response below.

Note

The read enumeration endpoints may return an inherited enumeration, for example an enumeration that does not exist for the exact specified project or target type, but is inherited from the global level or from a general enumeration. Such an inherited configuration cannot be modified (`PATCH`) or deleted (`DELETE`). If you want to override the inherited configuration, you need to create (`POST`) a new configuration for the desired project or target type.

#### CRUD operations for the enumerations

```bash
GET https://myhost/polarion/rest/v1/projects/myProject/enumerations/~/severity/testcase

```

- **Response Body**

```bash
{
  "links": {...},
  "data": {
    "type": "enumerations",
    "id": "drivepilot/~/severity/testcase",
    "attributes": {
      "options": [
        {
          "id": "transition",
          "name": "Transition",
          "color": "#E74C3C",
          "iconURL": "/polarion/icons/default/enums/severity_critical.gif"
        },
        {
          "id": "smoke",
          "name": "Smoke",
          "color": "#C0392B",
          "iconURL": "/polarion/icons/default/enums/severity_major.gif"
        },

        ...
      ],
      "enumName": "severity",
      "targetType": "testcase"
    },
    "links": {...}
  }
}

```

#### GET all enumerations in the Project context

Polarion has several types of enumerations:

- **Non-configurable enumerations**
  - Constant enumerations, for example `approval-status`
  - Special kind of enumerations, for example `variationType`
  - Object enumerations, for example `@project`
- **Configurable enumerations**
  - With target type, for example `documents/document-status`
  - Without target type, for example `work-item-type`
- **Custom enumerations**
  - For example almost any ID, but without "`/`"

All enumerations in the specified **Project** context can be fetched.

If a specific enumeration does not exist in the **Project** context but is available in the **Global** context, it will also be fetched.

- **Example Request:**

`GET https://myhost/polarion/rest/v1/projects/{projectId}/enumerations`

- **Response Body:**

```bash
{
  "links": {
    "self": "http://myhost/polarion/rest/v1/projects/elibrary/enumerations",
    "first": "http://myhost/polarion/rest/v1/projects/elibrary/enumerations?page%5Bnumber%5D=1",
    "last": "http://myhost/polarion/rest/v1/projects/elibrary/enumerations?page%5Bnumber%5D=1"
  },
  "data": [
    {
      "type": "enumerations",
      "id": "elibrary/documents/document-status/~",
      "attributes": {
        "options": [
          {
            "id": "draft",
            "name": "Draft",
            "iconURL": "/polarion/icons/default/enums/req_status_draft.gif",
            "minValue": 0
          },
          {
            "id": "active",
            "name": "Active",
            "iconURL": "/polarion/icons/default/enums/req_status_approved.gif",
            "minValue": 0
          },
          {
            "id": "obsolete",
            "name": "Obsolete",
            "iconURL": "/polarion/icons/default/enums/req_status_rejected.gif",
            "minValue": 0
          }
        ],
        "enumName": "document-status",
        "enumContext": "documents"
      },
      "links": {
        "self": "http://myhost/polarion/rest/v1/projects/elibrary/enumerations/documents/document-status/%7E"
      }
    },
    {
      "type": "enumerations",
      "id": "elibrary/documents/document-type/~",
      "attributes": {
        "options": [
          {
            "id": "productSpecification",
            "name": "Product Specification",
            "iconURL": "/polarion/icons/default/enums/document_package.png",
            "minValue": 0
          },
          {
            "id": "testSpecification",
            "name": "Test Specification",
            "iconURL": "/polarion/icons/default/enums/document_test_specification.png",
            "minValue": 0
          },
          {
            "id": "generic",
            "name": "Generic",
            "iconURL": "/polarion/icons/default/enums/document_generic.png",
            "default": true,
            "minValue": 0
          }
        ],
        "enumName": "document-type",
        "enumContext": "documents"
      },
      "links": {
        "self": "http://myhost/polarion/rest/v1/projects/elibrary/enumerations/documents/document-type/%7E"
      }
    }
  ]
}

```

#### GET all enumerations in the Global context

Only enumerations in the **Global** context will be fetched. However, the enumerations in specific **Projects** will not be fetched.

- **Example Request:**

`GET https://myhost/polarion/rest/v1/enumerations`

- **Response Body:**

```bash
{
  "links": {
    "self": "http://myhost/polarion/rest/v1/enumerations",
    "first": "http://myhost/polarion/rest/v1/enumerations?page%5Bnumber%5D=1",
    "last": "http://myhost/polarion/rest/v1/enumerations?page%5Bnumber%5D=1"
  },
  "data": [
    {
      "type": "enumerations",
      "id": "documents/document-status/~",
      "attributes": {
        "options": [
          {
            "id": "draft",
            "name": "Draft",
            "iconURL": "/polarion/icons/default/enums/req_status_draft.gif",
            "minValue": 0
          },
          {
            "id": "published",
            "name": "Published",
            "iconURL": "/polarion/icons/default/enums/req_status_approved.gif",
            "minValue": 0
          },
          {
            "id": "obsolete",
            "name": "Obsolete",
            "iconURL": "/polarion/icons/default/enums/req_status_rejected.gif",
            "minValue": 0
          }
        ],
        "enumName": "document-status",
        "enumContext": "documents"
      },
      "links": {
        "self": "http://myhost/polarion/rest/v1/enumerations/documents/document-status/%7E"
      }
    },
    {
      "type": "enumerations",
      "id": "documents/document-type/~",
      "attributes": {
        "options": [
          {
            "id": "req_specification",
            "name": "Requirements Specification",
            "iconURL": "/polarion/icons/default/enums/document_electrical_requirement2.png",
            "default": true,
            "minValue": 0
          },
          {
            "id": "generic",
            "name": "Generic",
            "iconURL": "/polarion/icons/default/enums/document_generic.png",
            "minValue": 0
          }
        ],
        "enumName": "document-type",
        "enumContext": "documents"
      },
      "links": {
        "self": "http://myhost/polarion/rest/v1/enumerations/documents/document-type/%7E"
      }
    }
  ]
}

```

### GET options related to enumeration fields

JAVA API fields of enumeration type are available in the REST API as either attributes or relationships (relationships are used for object-based enumerations). For every such field the REST API provides actions to get the current value of the field as a list of options, or to get available options that can be set to the field. In the following sections we will take a look at these actions in more detail.

Note

the enumeration field actions are supported even for object-based enumeration fields of JAVA API that are not yet supported as relationships, so they actually provide the only way to get their value using the REST API.

#### GET current options of an enumeration field

When a resource has an enumeration attribute (for example `severity` attribute of the `workitems` resource), the value of the attribute contains only the ID of the enumeration option (for example `critical`). To get the corresponding enumeration option with all its details, you can use the `getCurrentOptions` action available per field.

Note

The response always contains a list of options. For a single valued field this list has one item at most.

- **Get current options of an enumeration field**

```bash
GET https://myhost/polarion/rest/v1/projects/myProject/workitems/WI-123/fields/severity/actions/getCurrentOptions

```

- **Response Body**

```bash
{
  "links": {...},
  "data": [
    {
      "id": "severity",
      "name": "Critical",
      "color": "#C0392B",
      "iconURL": "/polarion/icons/default/enums/severity_critical.gif"
    }
  ]
}

```

#### GET available options of an enumeration field

For any enumeration field there is a `getAvailableOptions` action that returns the options fot the corresponding enumeration. This action is provided for a particular resource instance (for example for a particular `workitems` resource), and also independently of any resource instance of a given target type (for example for `workitems` of `testcase` type). The latter case is useful to get the available options when creating a new resource of a given type.

The response format is exactly the same as in the case of `getCurrentOptions`, so we will not repeat it in the examples below.

- **Examples**

`GET https://myhost/polarion/rest/v1/projects/myProject/workitems/WI-123/fields/severity/actions/getAvailableOptions`

Returns severity options available for the WI-123 **Work Item**.

If the type of the **Work Item** is `testcase`, the options are found in the testcase-specific enumeration of severities (or in inherited enumeration, if that specific one is not configured).

`GET https://myhost/polarion/rest/v1/projects/myProject/workitems/fields/severity/actions/getAvailableOptions?type=testcase`

Returns severity options available for **Work Items** of the `testcase` type in the `myProject` project.

The options are found in the testcase-specific enumeration of severities (or in inherited enumeration, if that specific one is not configured).

#### GET options with Link Rules

The **Response** of `GET` endpoints of enumerations for the `workitem-link-role` **ID** now include **Link Rules**.

##### Endpoints

- GET /projects/{projectID}/enumerations/{enumContext}/{enumName}/{targetType}
- GET /enumerations/{enumContext}/{enumName}/{targetType}
- GET /enumerations
- **Example Request**

`GET projects/elibrary/enumerations/~/workitem-link-role/~`

- **Response Body:**

```bash
{
  "links": {
    "self": "http://localhost:8888/polarion/rest/v1/projects/elibrary/enumerations/~/workitem-link-role/~"
  },
  "data": {
    "type": "enumerations",
    "id": "elibrary/~/workitem-link-role/~",
    "attributes": {
      "options": [
        {
          "id": "relates_to",
          "name": "relates to",
          "oppositeName": "is related to",
          "description": "A generic relation type, more concrete link should be used where possible",
          "default": true,
          "minValue": 0,
          "linkRules": [
            {
              "fromTypes": [
                "task"
              ],
              "sameType": true
            }
          ]
        },
        {
          "id": "parent",
          "name": "has parent",
          "oppositeName": "is parent of",
          "parent": true,
          "minValue": 0,
          "linkRules": [
            {
              "fromTypes": [
                "task",
                "changerequest"
              ],
              "toTypes": [
                "task"
              ]
            }
          ]
        }
      ],
      "enumName": "workitem-link-role"
    },
    "links": {
      "self": "http://localhost:8888/polarion/rest/v1/projects/elibrary/enumerations/~/workitem-link-role/~"
    }
  }
}

```

- **POST options with Link Rules**

To `POST` enumerations for the `workitem-link-role` **ID** along with **Link Rules**.

**Endpoints:**

- `POST /projects/{projectId}/enumerations`
- `POST /enumerations`
- **Request Body**

```bash
{
  "options": [
    {
      "id": "implements",
      "name": "implements",
      "oppositeName": "is implemented by",
      "description": "Used to link implementing Tasks and Issues to Work Packages and also Work Packages and Issues to Requirements.",
      "parent": true,
      "minValue": 0,
      "linkRules": [
        {
          "fromTypes": [
            "systemrequirement"
          ],
          "toTypes": [
            "softwarerequirement"
          ]
        },
        {
          "fromTypes": [
            "softwaretestcase"
          ],
          "toTypes": [
            "testcase"
          ]
        }
      ]
    },
    {
      "id": "depends_on",
      "name": "depends on",
      "oppositeName": "blocks",
      "description": "Used to mark a dependency between items.",
      "minValue": 0,
      "linkRules": [
        {
          "fromTypes": [
            "changerequest"
          ],
          "sameType": true
        }
      ]
    }
  ]
}

```

- **PATCH options with Link Rules:**

To post a **Link Role** attribute enum option via the `PATCH` enumeration API.

**Endpoints:**

- `PATCH /projects/{projectId}/enumerations`
- `PATCH /enumerations`
- **Request Body**

```bash
{
  "options": [
    {
      "id": "implements",
      "name": "implements",
      "oppositeName": "is implemented by",
      "description": "Used to link implementing Tasks and Issues to Work Packages and also Work Packages and Issues to Requirements.",
      "parent": true,
      "minValue": 0,
      "linkRules": [
        {
          "fromTypes": [
            "systemrequirement"
          ],
          "toTypes": [
            "softwarerequirement"
          ]
        },
        {
          "fromTypes": [
            "softwaretestcase"
          ],
          "toTypes": [
            "testcase"
          ]
        }
      ]
    },
    {
      "id": "depends_on",
      "name": "depends on",
      "oppositeName": "blocks",
      "description": "Used to mark a dependency between items.",
      "minValue": 0,
      "linkRules": [
        {
          "fromTypes": [
            "changerequest"
          ],
          "sameType": true
        }
      ]
    }
  ]
}

```

#### Read available enumeration icons

Available enumeration icons are also resources in the REST API. There are endpoints to get the default icons and custom icons (which can be configured per project or globally).

- **Example Request**

```bash
GET https://myhost/polarion/rest/v1/enumerations/defaulticons?fields[icons]=@all

```

- **Response Body**

```bash
{
  "links": {...},
  "data": [
    {
      "type": "icons",
      "id": "default/CMMI_stakeholders.gif",
      "attributes": {
        "path": "/polarion/icons/default/enums/CMMI_stakeholders.gif",
        "iconUrl": "https://myhost/polarion/icons/default/enums/CMMI_stakeholders.gif"
      },
      "links": {...}
    },
    ...
 ],
  "meta": {...}
}

```

#### Create enumeration icons

You can create icons for enumerations in the Project or Global context.

Note

You cannot create default icons. The size of icons cannot exceed 16x16 pixels.

- **Example Request**

```bash
POST https://myhost/polarion/rest/v1/enumerations/icons

```

- **Request Body**

```bash
{
  "data": [
    {
      "type": "icons"
    },
    {
      "type": "icons"
    }
  ]
}
files- (Array) - twitter.png, facebook.png (Icon files)

```

- **Response Body**

```bash
{
  "data": [
    {
      "type": "icons",
      "id": "group/facebook.png",
      "links": {
        "self": "https://myhost/polarion/rest/v1/enumerations/icons/facebook.png"
    }
    },
    {
      "type": "icons",
      "id": "group/twitter.png",
      "links": {
        "self": "https:/myhost/polarion/rest/v1/enumerations/icons/twitter.png"
      }
    }
  ]
}

```

#### Custom Enumeration Object Factories

Custom Object Enumerations are supported via Custom Object Enumeration factories.

Plugins providing Custom Object Enumerations must implement:

`com.polarion.platform.persistence.IEnumObjectFactory.getEnumOptionPrototypes()`

- If a custom plug-in that supplies the object factories implements `getEnumOptionPrototypes`, then the field that uses the custom object enumeration is displayed as an enum relationship.
- Otherwise, it is displayed as a string (the default). The default display is a list of strings if the custom field is a multi-value enum type.

**Response Body (GET)**: For a custom field where `custom_testruntemplate_single` is supplied but prototype information is not:

```bash
{
   "links":{
      "self":"http://localhost:8888/polarion/rest/v1/projects/base/workitems/WI-5?fields%5Bworkitems%5D=%40all"
   },
   "data":{
      "type":"workitems",
      "id":"base/WI-5",
      "attributes": {
         "id":"WI-5",
         "type":"task",
         "title":"Title before patch action.",
         "description": {
            "type":"text/plain",
            "value":"plain text description"
         },
         "severity":"normal",
         "priority":"50.0",
         "status":"open",
         "created":"2024-02-27T14:31:28.240Z",
         "updated":"2024-02-27T14:32:03.762Z",
         "custom_plantemplate_single":"base/testPlanTemplate"
      },
      "relationships":{
         "project":{
            "data":{
              "type":"projects",
              "id":"base"
            },
            "links":{
               "self":"http://localhost:8888/polarion/rest/v1/projects/base/workitems/WI-5/relationships/project"
            }
         },
         "author": {
            "data": {
               "type":"users",
               "id":"jUnitTester"
            },
            "links":{
               "self":"http://localhost:8888/polarion/rest/v1/projects/base/workitems/WI-5/relationships/author"
            }
         }
      },
      "links":{
         "self":"http://localhost:8888/polarion/rest/v1/projects/base/workitems/WI-5",
         "portal":"http://localhost:8888/polarion/redirect/project/base/workitem?id=WI-5""   "
      }
   }
}

```

**Response Body (GET):** For a custom field where both `custom_testruntemplate_single` and prototype information are supplied:

```bash
{
   "links":{
      "self":"http://localhost:8888/polarion/rest/v1/projects/base/workitems/WI-6?fields%5Bworkitems%5D=%40all"" "
   },
   "data":{
      "type":"workitems",
      "id":"base/WI-6",
      "attributes": {
         "id":"WI-6",
         "type":"task",
         "title":"Title before patch action.",
         "description": {
           "type":"text/plain",
           "value":"plain text description" 
         },
         "severity":"normal",
         "priority":"50.0",
         "status":"open",
         "created":"2024-02-27T14:46:43.550Z",
         "updated":"2024-02-27T14:47:27.335Z"
      },
      "relationships":{
         "project":{
            "data":{
               "type":"projects",
               "id":"base"
            },
            "links":{
               "self":"http://localhost:8888/polarion/rest/v1/projects/base/workitems/WI-6/relationships/project"
            }
         },
         "author":{
            "data":{
               "type":"users",
               "id":"jUnitTester"
            },
            "links":{
               "self":"http://localhost:8888/polarion/rest/v1/projects/base/workitems/WI-6/relationships/author"
            }
         },
         "assignee":{
            "links":{
               "self":"http://localhost:8888/polarion/rest/v1/projects/base/workitems/WI-6/relationships/assignee"
            }
         },
         "custom_testruntemplate_single": {
            "data":{
               "type":"testruns",
               "id":"base/testRunTemplate"
            },
            "links":{
               "self":"http://localhost:8888/polarion/rest/v1/projects/base/workitems/WI-6/relationships/custom_testruntemplate_single"
            }
         }
      },
      "links":{
         "self":"http://localhost:8888/polarion/rest/v1/projects/base/workitems/WI-6",
         "portal":"http://localhost:8888/polarion/redirect/project/base/workitem?id=WI-6"
      }
   }
}

```

If the custom object enumeration factory plug-in does not implement `getEnumOptionPrototypes`, then the payload must be supplied as a custom attribute for the custom enum option/custom enum options field, and the object is created/updated with the enum option(s).

**Request Body (POST):** If both a `custom field custom_plantemplate_single` and prototype information are **NOT** supplied:

```bash
{
   "data":[
      {
         "type":"workitems",
         "attributes":{
            "type":"task",
            "title":"WI-custom-single-enum-option-001",
            "description":{
               "type":"text/html",
               "value":"rich description"
            },
            "custom_plantemplate_single":"base/testPlanTemplate",
            "severity":"must_have",
            "priority":"71.0",
            "status":"closed",
            "resolution":"done",
            "initialEstimate":"3d",
            "timeSpent":"1d",
            "remainingEstimate":"2d",
            "dueDate":"2022-10-24"
         }
      }
   ]
}

```

If the custom object enumeration factory plug-in implements the `getEnumOptionPrototypes`, then the payload must be supplied as a relationship object for the custom enum options field and the object is created/updated with the enum option(s).

**Request Body (POST)**: if a `custom field custom_plantemplate_single` and prototypes information is not supplied.

```bash
{
   "data":[
      {
         "type":"workitems",
         "attributes":{
           "type":"task",
           "title":"WI-custom-single-enum-option-relship-001"
         },
         "relationships": {
            "assignee":{
               "data":[
                 {
                    "type":"users",
                    "id":"user1"
                  }
               ]
            },
            "author":{
              "data":{
                  "type":"users",
                  "id":"user1"
               }
            },
            "custom_testruntemplate_single": {
               "data":{
                  "type":"testruns",
                  "id":"base/testRunTemplate"
               }
            }
         }
      }
   ]
}

```

## Test Management

Test Management encompasses the artifacts which are instrumental in providing testing-related functionality. Endpoints are provided for the following artifacts:

- Test Run
- Test Run template
- Test Run attachment
- Test Run comment
- Test Parameter definition
- Test Parameter
- Test Record
- Test Record attachment
- Test Step Result
- Test Step Result attachment

The endpoint for fetching Test Runs and Test Run templates is the same. The value of the `templates` query parameter decides whether to fetch Test Run templates or Test Runs.

### Test Records

In Polarion, Test Records are created as a result of the execution of Test Cases. The execution action performs an automatic verification, creates artifacts (for example **Defects**), and relates them in the background. The execution action is currently not supported on the REST endpoints, so you have to use multiple endpoints to execute and create executed Test Records.

For example, if the **Defect** that is created when a Test Case fails (since the system is configured to automatically create a **Defect** if a Test Record fails), the **Defect** has to be created manually and specified as input relation while creating Test Records.

Test Records are zero indexed on the REST layer.

You have to specify the `testCaseRevision` parameter if you have to create or update a Test Record with a specific revision of the Test Case.

Test execution using signed context is not yet supported on the REST layer.

#### Create Test Records

- **Example Request**

```bash
POST https://myhost/polarion/rest/v1/projects/myProject/testruns/myTestRun/testrecords

```

- **Request Body**

```bash
{
  "data": [
    {
      "type": "testrecords",
      "attributes": {
        "comment": {
          "type": "text/html",
          "value": "my comment"
        },
        "duration": 0,
        "executed": "2023-01-01T00:00:00Z",
        "result": "failed"
      },
      "relationships": {
        "defect": {
          "data": {
            "type": "workitems",
            "id": "myProject/myDefectId"
          }
        },
        "executedBy": {
          "data": {
            "type": "users",
            "id": "MyUserId"
          }
        },
        "testCase": {
          "data": {
            "type": "workitems",
            "id": "myProject/myTestCase"
          }
        }
      }
    }
  ]
}

```

- **Response Body**

```bash
{
  "data": [
    {
      "type": "testrecords",
      "id": "myProject/myTestRun/myProject/myTestcase/0",
      "links": {
        "self": "https://myHost/polarion/rest/v1/projects/myProject/testruns/myTestRun/testrecords/myProject/myTestcase/0"
      }
    }
  ]
}

```

### Test Record Deletion

Test Records can be deleted via the REST API. Only a single Test Record can be deleted at a time.

### Example Request

`http://localhost:8888/polarion/rest/v1/projects/ProjectID/testruns/TestRunIde/testrecords/TestCaseProject/TestCaseId/0`

### Response Body

```bash
{
  "errors": [
    {
      "status": "404",
      "title": "Not Found",
      "detail": "Test Record 'ProjectID/TestRunIde/TestCaseProject/TestCaseId/0' was not found.",
      "source": {
        "pointer": null,
        "parameter": null,
        "resource": {
          "id": "ProjectID/TestRunIde/TestCaseProject/TestCaseId/0",
          "type": "testrecords"
        }
      }
    }
  ]
}

```

### Test Parameter Definition

Test Parameter Definitions define the name of the parameter. Test Parameter Definitions can be added to a Library, Project, and also to Test Runs.

### Test Parameter

The value added to a Test Parameter Definition is the Test Parameter. Test Parameters can be added to Test Runs and Test Records.

### Import Test Results

The REST layer provides endpoints to import Test Results in XUnit and Excel formats. Since the import process can take a long time to complete, the process is run as a Polarion Job (Asynchronous Request Reply pattern).

The complete process is executed as a two-step process:

1. The import process is started and you receive the `202 Accepted` response if the process started successfully. The response also sends the `jobId` of the Polarion Job which is started, along with a link to the job log.
1. The state and status of the import process has to be obtained through the `jobs/{jobId}` endpoint.

#### Example Request (beginning of import)

```bash
POST https://myhost/polarion/rest/v1/projects/myProject/testruns/myTestRun/actions/importXUnitTestResults

```

- **Response Body (202 Accepted)**

```bash
{
  "data": {
    "type": "jobs",
    "id": "8dd9cae6-927aec93-66855dab-098a0106",
    "attributes": {
      "jobId": "8dd9cae6-927aec93-66855dab-098a0106",
      "name": "Import of test results",
      "state": "RUNNING"
    },
    "links": {
      "self": "https://myhost/polarion/rest/v1/jobs/8dd9cae6-927aec93-66855dab-098a0106",
      "log": "https://myhost/polarion/job-report?jobId=8dd9cae6-927aec93-66855dab-098a0106"
    }
  }
}

```

- **Example Request (import job state and status)**

```bash
GET https://myhost/polarion/rest/v1/jobs/8dd9cae6-927aec93-66855dab-098a0106

```

- **Response Body (200 OK)**

```bash
{
  "links": {
    "self": "https://myhost/polarion/rest/v1/jobs/8dd9cae6-927aec93-66855dab-098a0106"
  },
  "data": {
    "type": "jobs",
    "id": "8dd9cae6-927aec93-66855dab-098a0106",
    "attributes": {
      "jobId": "8dd9cae6-927aec93-66855dab-098a0106",
      "name": "Import of test results",
      "state": "FINISHED",
      "status": {
        "type": "OK"
      }
    },
    "links": {
      "self": "https://myhost/polarion/rest/v1/jobs/8dd9cae6-927aec93-66855dab-098a0106",
      "log": "https://myhost/polarion/job-report?jobId=8dd9cae6-927aec93-66855dab-098a0106"
    }
  }
}

```

### Export Tests to Excel

You can export Test Results using the `exportTestsToExcel` endpoint. You can control the output of the export by supplying the following parameters in the Request Body:

- `query` (a Lucene query to select Test Cases to export)
- `sortby` (the sort criteria for the content of the export)
- `template` (the template to use for the export)

#### Example Request (beginning of export)

```bash
POST https://myhost/polarion/rest/v1/projects/myProject/testruns/myTestRun/actions/exportTestsToExcel

```

- **Response Body (202 Accepted)**

```bash
{
  "data": {
    "type": "jobs",
    "id": "8e496cc1-927aec93-7f1efe40-eed209a9",
    "attributes": {
      "jobId": "8e496cc1-927aec93-7f1efe40-eed209a9",
      "name": "Export (xlsx: Microsoft Excel Tests)",
      "state": "RUNNING"
    },
    "links": {
      "self": "https://myhost/polarion/rest/v1/jobs/8e496cc1-927aec93-7f1efe40-eed209a9",
      "log": "https://myhost/polarion/job-report?jobId=8e496cc1-927aec93-7f1efe40-eed209a9"
    }
  }
}

```

- **Example Request (export job state and status)**

```bash
GET https://myhost/polarion/rest/v1/jobs/8e496cc1-927aec93-7f1efe40-eed209a9

```

- **Response Body (200 OK)**

```bash
{
  "links": {
    "self": "https://myhost/polarion/rest/v1/jobs/8e496cc1-927aec93-7f1efe40-eed209a9"
  },
  "data": {
    "type": "jobs",
    "id": "8e496cc1-927aec93-7f1efe40-eed209a9",
    "attributes": {
      "jobId": "8e496cc1-927aec93-7f1efe40-eed209a9",
      "name": "Export (xlsx: Microsoft Excel Tests)",
      "state": "FINISHED",
      "status": {
        "type": "OK"
      }
    },
    "links": {
      "self": "https://myhost/polarion/rest/v1/jobs/8e496cc1-927aec93-7f1efe40-eed209a9",
      "log": "https://myhost/polarion/job-report?jobId=8e496cc1-927aec93-7f1efe40-eed209a9",
      "downloads": [
        "https://myhost/polarion/rest/v1/jobs/8e496cc1-927aec93-7f1efe40-eed209a9/filename/tests.xlsx"
      ]
    }
  }
}

```

- **Example Request (to download the exported content)**

```bash
GET https://myhost/polarion/rest/v1/jobs/8e496cc1-927aec93-7f1efe40-eed209a9/actions/download/tests.xlsx

```

## Page parameter values

Polarion **Info Pages** and **LiveReport Pages** can be accessed through the Polarion REST API to read and update their properties and default parameter values. (Accessing or modifying the page content is not possible. Working with **Page** attachments is possible, see [Attachments](advanced-concepts.html#attachments).)

### Get Page parameters

**Page** parameters show among fields as special attributes with the `parameter_` prefix, but only if the particular page parameter has a non-empty default value set and it was explicitly requested to be fetched, or `fields[pages]=@all` was used.

- **Example Request (export job state and status)**

```bash
GET https://myhost/polarion/rest/v1/projects/myProject/spaces/_default/pages/MyInfoPage?fields[pages]=@basic,parameter_severity

```

- **Response Body**

```bash
{
    "links": {
        "self": "https://myhost/polarion/rest/v1/projects/myProject/spaces/_default/pages/MyPage?fields%5Bpages%5D=%40basic%2Cparameter_severity"
    },
    "data": {
        "type": "pages",
        "id": "myProject/_default/MyPage",
        "attributes": {
            "pageName": "MyPage",
            "spaceId": "_default",
            "title": "My Report Page",
            "parameter_severity": "basic"
        },
        "relationships": {...},
        "links": {...}
    }
}

```

### Update Page parameters

The following example shows the parameter with `severity` ID (presented as the `parameter_severity` attribute) and its current default value. This default value can be modified via the REST API using a `PATCH` request to the **Page** resource. It can be used to influence the **LiveReport** or **Info Page** content if the parameter value is used by the **Page's** widgets or scripts.

- **Example Request**

```bash
PATCH https://myhost/polarion/rest/v1/projects/myProject/spaces/_default/pages/MyPage

```

- **Response Body**

```bash
{
  "data": {
    "type": "pages",
    "id": "myProject/_default/MyPage",
    "attributes": {
      "parameter_severity": "major"
    }
  }
}

```

## Collections

### Collections overview

**Collections** support parallel development activities, audit and regulatory compliance, and advanced reuse scenarios. The Polarion REST API provides ways to read, create, and manipulate **Collections**.

**Collections** behave just like other resources, but there are a few special cases, such as closing and reopening a **Collection**.

### Close a Collection

You can **Close** a **Collection** with the endpoint below:

- **Example Request**

```bash
POST /polarion/rest/v1/projects/myProject/collections/{collectionId}/actions/close

```

Add the `projectId` and `collectionId` as parameters.

- **Response:**

Collection is closed.

`204 - No Content`

Note

- Hitting the endpoint for an already closed **Collection** returns a `204 - No Content` response.
- All **Documents** inside the **Collection** should have a revision other than `HEAD`. Otherwise, a `400 BAD REQUEST` error is thrown, and the following message appears: `Change all HEAD Documents to a baseline or revision before closing the Collection`.
- Any attempt to change a closed **Collection** (except reopening) returns `400` with the error message.

### Reopen a Collection

You can **Reopen** a **Collection** with the endpoint below:

- **Example Request**

```bash
POST /polarion/rest/v1/projects/myProject/collections/{collectionId}/actions/reopen

```

Add the `projectId` and `collectionId` as parameters.

- **Response:** Collection is reopened.

`204 - No Content`

Note

- Hitting the endpoint for an already opened **Collection** returns `204 - No Content` response.

### Reuse a Collection

You can **Reuse** a **Collection** using the endpoint below:

- **Example Request:**

```bash
POST https://hostname/polarion/rest/v1/projects/myproject/collections/%7BcollectionId%7D/targetProjectId/%7BtargetProjectId%7D/actions/reuse 

```

Add the `projectId`, `collectionId`, and `targetProjectId` as parameters.

- **Response:** Collection is reused.

`204 - No Content`

## License overview

The newly introduced licensing endpoints are designed to align with the UI and provide several key features.

Users can assign default concurrent licenses, including both base and addon types, as well as create and delete license groups. The system allows for checking license usage, including detailed usage information and usage limits. Additionally, users can assign licenses to specific individuals and view an overview of assigned slots for particular license types.

Note

Support for cloud-based [Polarion X](https://www.siemens.com/en-us/products/polarion/polarion-x/) is limited; some License endpoints are not supported.

### License

#### GET /license

Retrieve a single product license with information about licensing limits and usage.

- **Response:**

```bash
{
  "type": "license",
  "id": "license",
  "attributes": {
    "limits": {
      "projects": {
        "limit": 100,
        "currentCount": 9
      },
      "documentsAndPages": {
        "limit": 10000,
        "currentCount": 235
      },
      "workitems": {
        "limit": 100000,
        "currentCount": 12548
      }
    }
  }
}

```

#### PATCH /license

Updates the product license.

Note

This endpoint is currently not supported in [Polarion X](https://www.siemens.com/en-us/products/polarion/polarion-x/).

- **Request Body:**

```bash
{
  "data": {
    "type": "license",
    "id": "license",
    "relationships": {
      "defaultAddOnLicenseSlots": {
        "data": [
          {
            "type": "license_slots",
            "id": "LicenseType/ModelType/GroupType"
          }
        ]
      },
      "defaultBaseLicenseSlot": {
        "data": {
          "type": "license_slots",
          "id": "LicenseType/ModelType/GroupType"
        }
      }
    }
  }
}

```

### License Slots

#### GET /license/types/{typeId}/slots

Retrieve all available License Slots of a given license type.

For example, `ALM named, ALM concurrent, ALM concurrent group DepartmentA`.

Note

This endpoint is currently not supported in [Polarion X](https://www.siemens.com/en-us/products/polarion/polarion-x/).

- **Example Request:**

`GET /license/ALM/slots`

- **Response:**

```bash
{
  "data": [
    {
      "type": "license_slots",
      "id": "ALM/concurrent/DepartmentA",
      "attributes": {
        "concurrent": true,
        "group": "DepartmentA",
        "total": 100,
        "configured": 90,
        "free": 10,
        "peak": 55
      }
    }
  ]
}

```

#### GET /license/types/{typeId}/slots/{model}/{group}

Retrieve the available License Slot of a given license type.

For example, `ALM concurrent group DepartmentA`.

Note

This endpoint is currently not supported in [Polarion X](https://www.siemens.com/en-us/products/polarion/polarion-x/).

- **Example Request:**

`GET /license/types/ALM/slots/concurrent/DepartmentA`

- **Response:**

```bash
{
  "data": {
    "type": "license_slots",
    "id": "ALM/concurrent/DepartmentA",
    "attributes": {
      "model": "concurrent",
      "group": "DepartmentA",
      "total": 100,
      "configured": 90,
      "free": 10,
      "peak": 55,
      "expirationDate": "2026-01-05"
    }
  },
  "links": {
    "self": "https://myhost/polarion/rest/v1/license/types/ALM/slots/concurrent/DepartmentA"
  }
}

```

#### POST /license/types/{typeId}/slots

Create a single or multiple groups of a given license type. For example, create `ALM Concurrent group DepartmentA`.

Note

License Groups are called `slots` in REST.

This endpoint is currently not supported in [Polarion X](https://www.siemens.com/en-us/products/polarion/polarion-x/).

- **Example Request:**

`POST /license/types/{typeId}/slots`

- **Request Body:**

```bash
POST /license/types/ALM/slots
{
  "data": [
    {
      "type": "license_slots",
      "attributes": {
        "group": "DepartmentA",
        "size": 20
      }
    },
    {
      "type": "license_slots",
      "attributes": {
        "group": "DepartmentB",
        "size": 20
      }
    }
  ]
}

```

- **Response:**

```bash
{
  "data": [
    {
      "type": "license_slots",
      "id": "ALM/concurrent/DepartmentA",
      "links": {
        "self": "server-host-name/application-path/license/types/ALM/slots/concurrent/DepartmentA"
      }
    },
    {
      "type": "license_slots",
      "id": "ALM/concurrent/DepartmentB",
      "links": {
        "self": "server-host-name/application-path/license/types/ALM/slots/concurrent/DepartmentB"
      }
    }
  ]
}

```

#### DELETE /license/types/{typeId}/slots

Delete an available License Slot of a given license type.

For example, `ALM concurrent group DepartmentA`.

Note

This endpoint is currently not supported in [Polarion X](https://www.siemens.com/en-us/products/polarion/polarion-x/).

- **Example Request:**

`DELETE /license/types/{typeId}/slots`

- **Request Body:**

```bash
{
  "data": [
    {
      "type": "license_slots",
      "id": "ALM/concurrent/demo"
    },
    {
      "type": "license_slots",
      "id": "ALM/concurrent/demo"
    }
  ]
}

```

### License Assignment

#### GET /license/assignments

Retrieve the license assignments for all the available users.

Note

This endpoint is currently not supported in [Polarion X](https://www.siemens.com/en-us/products/polarion/polarion-x/).

- **Example Request:**

`GET /license/assignments`

- **Response:**

```bash
  "data": [
    {
      "type": "license_assignments",
      "id": "admin",
      "attributes": {
        "status": "loggedIn"
      },
      "relationships": {
        "baseSlot": {
          "data": {
            "type": "license_slots",
            "id": "QA/concurrent/~"
          }
        },
        "user": {
          "data": {
            "type": "users",
            "id": "admin"
          }
        }
      }
    },
    {
      "type": "license_assignments",
      "id": "steve",
      "attributes": {
        "status": "expiring"
      },
      "relationships": {
        "baseSlot": {
          "data": {
            "type": "license_slots",
            "id": "ALM/concurrent/DepartmentA"
          }
        },
        "addOnSlots": {
          "data": {
            "type": "license_slots",
            "id": "Safe/concurrent/DepartmentS"
          }
        },
        "user": {
          "data": {
            "type": "users",
            "id": "steve"
          }
        }
      }...
    }
  ]

```

#### GET /license/assignments/{userId}

Retrieve the available license assignment of a given user.

Note

This endpoint is currently not supported in [Polarion X](https://www.siemens.com/en-us/products/polarion/polarion-x/).

- **Request:**

`GET /license/assignments/admin`

- **Response:**

```bash
{
  "data": {
    "type": "license_assignments",
    "id": "admin",
    "attributes": {
      "status": "LOGGED_IN"
    },
    "relationships": {
      "baseSlot": {
        "data": {
          "type": "license_slots",
          "id": "ALM/concurrent/~"
        }
      },
      "addOnSlots": {
        "data": [
          {
            "type": "license_slots",
            "id": "Variants/named/~"
          }
        ]
      },
      "user": {
        "data": {
          "type": "users",
          "id": "admin"
        }
      }
    },
    "links": {
      "self": "https://myhost/polarion/rest/v1/license/assignments/admin"
    }
  }
}

```

#### PATCH /license/assignments

Automatically update the license assignments for multiple users, locking each updated user for a default time of 24 hours.

Note

This endpoint is currently not supported in [Polarion X](https://www.siemens.com/en-us/products/polarion/polarion-x/).

- **Request:**

```bash
{
  "data": [
    {
      "type": "license_assignments",
      "id": "userId",
      "relationships": {
        "baseLicenseSlot": {
          "data": {
            "type": "license_slots",
            "id": "LicenseType/ModelType/GroupId"
          }
        },
        "addOnLicenseSlots": [
          {
            "data": {
              "type": "license_slots",
              "id": "LicenseType/ModelType/GroupId"
            }
          }
        ]
      }
    },
    {
      "type": "license_assignments",
      "id": "userId",
      "relationships": {
        "baseLicenseSlot": {
          "data": {
            "type": "license_slots",
            "id": "LicenseType/ModelType/GroupId"
          }
        },
        "addOnLicenseSlots": [
          {
            "data": {
              "type": "license_slots",
              "id": "LicenseType/ModelType/GroupId"
            }
          }
        ]
      }
    }
  ]
}

```

- **Response:**

`204 - No Content`

## Jobs

The existing endpoints related to **Jobs** have been enhanced to provide additional capability to view **Job** information for specific **Projects**, execute **Jobs** through all the supported parameters, and download the log content to analyze the **Jobs**.

### GET/jobs

To list all available **Jobs** at the **Global** level. It can also be used to get the specified **Jobs** at the **Project** level when a `projectId` is defined.

- **Request:**

`GET/jobs`

- **Response:**

```bash
{
  "data": [
    {
      "type": "jobs",
      "id": "91335d53-52e2-459f-9619-73ef3a3d4ff0",
      "attributes": {
        "jobId": "91335d53-52e2-459f-9619-73ef3a3d4ff0",
        "name": "Attachment Indexer",
        "state": "FINISHED",
        "status": {
          "type": "OK"
        }
      },
      "links": {
        "self": "https://myhost/polarion/rest/v1/jobs/91335d53-52e2-459f-9619-73ef3a3d4ff0",
        "log": "https://myhost/polarion/job-report?jobId=91335d53-52e2-459f-9619-73ef3a3d4ff0"
      }
    },
    {
      "type": "jobs",
      "id": "264aead0-0913-4300-a1d8-524b82b1dd22",
      "attributes": {
        "jobId": "264aead0-0913-4300-a1d8-524b82b1dd22",
        "name": "DB History Creator",
        "state": "FINISHED",
        "status": {
          "type": "OK"
        }
      },
      "links": {
        "self": "https://myhost/polarion/rest/v1/jobs/264aead0-0913-4300-a1d8-524b82b1dd22",
        "log": "https://myhost/polarion/job-report?jobId=264aead0-0913-4300-a1d8-524b82b1dd22"
      }
    },
    {
      "type": "jobs",
      "id": "79830ece-4c50-4c80-a06d-8af133ab7d56",
      "attributes": {
        "jobId": "79830ece-4c50-4c80-a06d-8af133ab7d56",
        "name": "Live Plan Chart Update",
        "state": "FINISHED",
        "status": {
          "type": "OK"
        }
      },
      "links": {
        "self": "https://myhost/polarion/rest/v1/jobs/79830ece-4c50-4c80-a06d-8af133ab7d56",
        "log": "https://myhost/polarion/job-report?jobId=79830ece-4c50-4c80-a06d-8af133ab7d56"
      }
    },
    {
      "type": "jobs",
      "id": "d60d271c-35cc-40bf-b3dc-ccd7674327a2",
      "attributes": {
        "jobId": "d60d271c-35cc-40bf-b3dc-ccd7674327a2",
        "name": "Live Plan Chart Update - Index Refresh",
        "state": "FINISHED",
        "status": {
          "type": "OK"
        }
      },
      "links": {
        "self": "https://myhost/polarion/rest/v1/jobs/d60d271c-35cc-40bf-b3dc-ccd7674327a2",
        "log": "https://myhost/polarion/job-report?jobId=d60d271c-35cc-40bf-b3dc-ccd7674327a2"
      }
    }
  ]
}

```

### POST /jobs/actions/execute

Executes a **Job** with or without parameters.

- **Request:**

`POST /jobs/actions/execute`

- **Request Body:**

```bash
{
  "jobId": "execute.command",
  "name": "Polarion Diagnostic Tool",
  "scope": "system",
  "params": {
    "command": "$[com.polarion.home]/diagtool/run$[shellExtension]",
    "arguments": [
      "-sendResults",
      "no",
      "-resultsDir",
      "$[jobDir]"
    ]
  }
}

```

- **Response Body:**

```bash
{
  "data": {
    "type": "jobs",
    "id": "ebb74cc2-ebbd-47a6-bd99-96ce8862b466",
    "attributes": {
      "jobId": "ebb74cc2-ebbd-47a6-bd99-96ce8862b466",
      "name": "Polarion Diagnostic Tool",
      "state": "RUNNING"
    },
    "links": {
      "self": "https://myhost/polarion/rest/v1/jobs/ebb74cc2-ebbd-47a6-bd99-96ce8862b466",
      "log": "https://myhost/polarion/job-report?jobId=ebb74cc2-ebbd-47a6-bd99-96ce8862b466"
    }
  }
}

```

### GET /jobs/{jobId}/log/content

To download the **Log** content of a specific **Job**.

- **Request:**

`GET /jobs/98021cc5-f9d3-4e41-aa81-8486549787f8/log/content`

- **Response:**

```bash
200

```

## Metadata

### Get global fields metadata

To get all fields of a resource type or its target type at the **Global**. All built-in and custom fields that the current user can read, even inherited ones, are included.

- **Endpoint:**

```bash
GET /actions/getFieldsMetadata?resourceType={resourceType}&targetType={targetType}

```

- **Response:**

```bash
{
  "data": {
    "attributes": {
      "primitiveField": {
        "label": "field-label",
        "type": {
          "kind": {
            "enum": [
              "string"
            ]
          }
        }
      },
      "enumField": {
        "label": "field-label",
        "type": {
          "kind": "enumeration",
          "enumContext": "enum-context",
          "enumName": "enum-name"
        }
      },
      "structField": {
        "label": "field-label",
        "type": {
          "kind": "structure",
          "structureName": "structureId"
        }
      },
      "listField": {
        "label": "field-label",
        "type": {
          "itemType": {
            "kind": {
              "enum": [
                "string"
              ]
            }
          },
          "kind": "list"
        }
      }
    },
    "relationships": {
      "relationshipField": {
        "label": "field-label",
        "type": {
          "kind": "relationship",
          "multi": true,
          "targetResourceTypes": [
            "resource-type"
          ]
        }
      }
    }
  }
}

```

### Get Project fields metadata

To get all fields of a resource type or its target type at **Project** level. All built-in and custom fields that the current user can read, even inherited ones, are included.

- **Endpoints:**

```bash
GET /projects/{projectId}/actions/getFieldsMetadata?resourceType={resourceType}&targetType={targetType}

```

- **Response:**

```bash
{
  "data": {
    "attributes": {
      "primitiveField": {
        "label": "field-label",
        "type": {
          "kind": {
            "enum": [
              "string"
            ]
          }
        }
      },
      "enumField": {
        "label": "field-label",
        "type": {
          "kind": "enumeration",
          "enumContext": "enum-context",
          "enumName": "enum-name"
        }
      },
      "structField": {
        "label": "field-label",
        "type": {
          "kind": "structure",
          "structureName": "structureId"
        }
      },
      "listField": {
        "label": "field-label",
        "type": {
          "itemType": {
            "kind": {
              "enum": [
                "string"
              ]
            }
          },
          "kind": "list"
        }
      }
    },
    "relationships": {
      "relationshipField": {
        "label": "field-label",
        "type": {
          "kind": "relationship",
          "multi": true,
          "targetResourceTypes": [
            "resource-type"
          ]
        }
      }
    }
  }
}

```

### Get instance level field metadata

To retrieve all fields of a specified instance.

This includes all built-in and custom fields that the current user can read, including inherited ones.

This feature is currently supported only for the following resource types with custom fields:

**Work Items, Collections, Documents, Plans, Test Runs, Test Records**.

- **Example Endpoints**

`GET /projects/{projectId}/workitems/{workItemId}/actions/getFieldsMetadata`

`GET /projects/{projectId}/testruns/{testRunId}/actions/getFieldsMetadata`

`GET /projects/{projectId}/testruns/{testRunId}/testrecords/{testCaseProjectId}/{testCaseId}/{iteration}/actions/getFieldsMetadata`

`GET /projects/{projectId}/spaces/{spaceId}/documents/{documentName}/actions/getFieldsMetadata`

`GET /projects/{projectId}/collections/{collectionId}/action/getFieldsMetadata`

`GET /projects/{projectId}/plans/{planId}/actions/getFieldsMetadata`

- **Response:**

```bash
{
  "data": {
    "attributes": {
      "primitiveField": {
        "label": "field-label",
        "defaultValue": "default-string",
        "description": "description-string",
        "type": {
          "kind": {
            "enum": [
              "string"
            ]
          }
        }
      },
      "enumField": {
        "label": "field-label",
        "type": {
          "kind": "enumeration",
          "enumContext": "enum-context",
          "enumName": "enum-name"
        },
        "dependsOn": "depends-on-field-id",
        "mapping": "dependency-mapping-xml-id"
      },
      "structField": {
        "label": "field-label",
        "type": {
          "kind": "structure",
          "structureName": "structureId"
        }
      },
      "listField": {
        "label": "field-label",
        "type": {
          "kind": {
            "enum": [
              "string"
            ]
          },
          "multi": "true"
        }
      }
    },
    "relationships": {
      "relationshipField": {
        "label": "field-label",
        "type": {
          "kind": "relationship",
          "multi": true,
          "targetResourceTypes": [
            "resource-type"
          ]
        }
      }
    }
  }
}

```

### Get global metadata

You can perform a `GET` request to retrieve the **Global**-level, read-only metadata.

**Resource type:** `metadata`

**Resource ID:** `metadata`

- **Endpoint:**

```bash
GET /metadata

```

- **Response:**

```bash
{
  "data": {
    "type": "metadata",
    "id": "metadata",
    "attributes": {
      "apiProperties": {
        "defaultPageSize": 100,
        "bodySizeLimit": 2097152,
        "maxIncludedSize": 500,
        "maxPageSize": 200,
        "maxRelationshipSize": 100
      },
      "build": "20250613-1404-master-e594c717",
      "cluster": "cluster1",
      "logoUrl": "/images/logos/repo_login_logo.png",
      "node": "node2",
      "timezone": "+05:30",
      "version": "3.25.12"
    }
  }
}

```

## Custom fields configuration

### Fetching custom fields

#### Get Project custom fields

To retrieve a custom field's configuration for a specified resource type and target type within a given **Project**. This involves identifying the project parameters (if applicable), specifying the resource type and target type, and using the API to fetch the associated custom field definitions.

- **Endpoint:**

`GET /projects/{projectId}/customfields/{resourceType}/{targetType}`

- **Response Body:**

```bash
{
  "links": {
    "self": "http://myhost/polarion/rest/v1/projects/base/customfields/workitems/task"
  },
  "data": {
    "type": "customfields",
    "id": "base/workitems/task",
    "attributes": {
      "fields": [{
          "id": "user",
          "name": "Test User",
          "type": {
            "kind": "relationship",
            "targetResourceTypes": ["users"],
              "multi": true
          }
        },
    } {
          "id": "plan",
          "name": "Test Plan",
          "type": {
            "kind": "relationship",
            "targetResourceTypes": ["plans"]
          }
       }, {
          "id": "enumField",
          "name": "Enum Field",
          "description": "Sample Enumeration for Custom Field",
          "required": true,
          "type": {
            "kind": "enumeration",
            "enumName": "@timePoint",
            "multi": true
          }
       }, {
          "id": "document_status",
          "name": "Document Status",
          "type": {
            "kind": "enumeration",
            "enumName": "document-status",
            "enumContext": "documents"
          }
       }, {
          "id": "fieldMapping",
          "name": "Mapping Field",
          "type": {
            "kind": "enumeration",
            "enumName": "@plan"
          },
          "dependsOn": "document_status"
       }]
    },
    "links": {
      "self": "http://myhost/polarion/rest/v1/projects/base/customfields/workitems/task"
    }
  }
}

```

#### Get Global custom fields

To retrieve a custom field's configuration for a specified resource type and target type within the **Global** context. This involves specifying the resource type and target type, and using the API to fetch the associated custom field definitions.

- **Endpoint:**

`GET /customfields/{resourceType}/{targetType}`

- **Response Body:**

```bash
{
  "links": {
    "self": "http://myhost/polarion/rest/v1/customfields/workitems/task"
  },
  "data": {
    "type": "customfields",
    "id": "workitems/task",
    "attributes": {
      "fields": [{
          "id": "user",
          "name": "Test User",
          "type": {
            "kind": "relationship",
            "targetResourceTypes": ["users"],
            "multi": true
          }
        }
    } {
          "id": "plan",
          "name": "Test Plan",
          "type": {
            "kind": "relationship",
            "targetResourceTypes": ["plans"]
          }
       }, {
          "id": "enumField",
          "name": "Enum Field",
          "description": "Sample Enumeration for Custom Field",
          "required": true,
          "type": {
            "kind": "enumeration",
            "enumName": "@timePoint",
            "multi": true
          }
       }, {
          "id": "document_status",
          "name": "Document Status",
          "type": {
            "kind": "enumeration",
            "enumName": "document-status",
            "enumContext": "documents"
          }
       }, {
          "id": "fieldMapping",
          "name": "Mapping Field",
          "type": {
            "kind": "enumeration",
            "enumName": "@plan"
          },
          "dependsOn": "document_status"
       }]
    },
    "links": {
      "self": "http://myhost/polarion/rest/v1/customfields/workitems/task"
    }
  }
}

```

### Creating custom fields

#### Creating Project custom fields

To create a custom field's configuration for a specific resource type within a given **Project**. This involves identifying the **Project** parameters, specifying the resource type and target type, and using the API to define the associated custom field.

- **Endpoint:**

`POST /projects/{projectId}/customfields`

- **Request Body:**

```bash
{
  "data": [
    {
      "type": "customfields",
      "attributes": {
        "fields": [
          {
            "id": "table",
            "name": "Sample Table",
            "description": "Sample Table",
            "required": false,
            "type": {
              "kind": "structure",
              "structureName": "Table"
            },
            "parameters": [
              {
                "key": "specNumber",
                "name": "Specification Number",
                "title": "Number of the specification"
              }
            ]
          }
        ],
        "resourceType": "workitems",
        "targetType": "issue"
      }
    }
  ]
}

```

- **Response:**

```bash
201 CREATED
{
  "data": [
    {
      "type": "customfields",
      "id": "elibrary/workitems/issue",
      "links": {
        "self": "http://myhost/polarion/rest/v1/projects/elibrary/customfields/workitems/issue"
      }
    }
  ]
}

```

#### Creating Global custom fields

To create a custom field's configuration for a specific resource type within a given default repository. This involves identifying and specifying the resource type and target type, and using the API to define the associated custom field.

- **Endpoint:**

`POST /customfields`

- **Request Body:**

```bash
{
  "data": [
    {
      "type": "customfields",
      "attributes": {
        "fields": [
          {
            "id": "table",
            "name": "Sample Table",
            "description": "Sample Table",
            "required": false,
            "type": {
              "kind": "structure",
              "structureName": "Table"
            },
            "parameters": [
              {
                "key": "specNumber",
                "name": "Specification Number",
                "title": "Number of the specification"
              }
            ]
          }
        ],
        "resourceType": "workitems",
        "targetType": "issue"
      }
    }
  ]
}

```

- **Response:**

```bash
201 CREATED
{
  "data": [
    {
      "type": "customfields",
      "id": "workitems/issue",
      "links": {
        "self": "http://myhost/polarion/rest/v1/customfields/workitems/issue"
      }
    }
  ]
}

```

### Updating custom fields

#### Updating Project custom fields

To update a custom field's configuration for a specified resource type and target type within a given **Project**. This involves identifying the target scope, specifying the resource type and target type, and providing a list of updated field definitions in the Request Body to modify the corresponding custom fields.

- **Endpoint:**

`PATCH /projects/{projectId}/customfields/{resourceType}/{targetType}`

- **Request Body:**

```bash
{
  "data": [
    {
      "type": "customfields",
      "attributes": {
        "fields": [
          {
            "id": "table",
            "name": "Sample Table",
            "description": "Sample Table",
            "required": false,
            "type": {
              "kind": "structure",
              "structureName": "Table"
            },
            "parameters": [
              {
                "key": "specNumber",
                "name": "Specification Number",
                "title": "Number of the specification"
              }
            ]
          }
        ]
      }
    }
  ]
}

```

- **Response:**

```bash
204 - No Content

```

#### Update Global custom fields

To update a custom field's configuration for a specified resource type and target type within a given **Global** context. This involves identifying the target scope, specifying the resource type and target type, and providing a list of updated field definitions in the Request Body to modify the corresponding custom fields.

- **Endpoint:**

`PATCH /customfields/{resourceType}/{targetType}`

- **Request Body:**

```bash
{
  "data": [
    {
      "type": "customfields",
      "attributes": {
        "fields": [
          {
            "id": "table",
            "name": "Sample Table",
            "description": "Sample Table",
            "required": false,
            "type": {
              "kind": "structure",
              "structureName": "Table"
            },
            "parameters": [
              {
                "key": "specNumber",
                "name": "Specification Number",
                "title": "Number of the specification"
              }
            ]
          }
        ]
      }
    }
  ]
}

```

- **Response:**

```bash
204 - No Content

```

### Deleting custom fields

#### Delete Project custom fields

To delete a custom field's configuration for a specified resource type and target type within a given **Project**. This involves identifying the target scope, and specifying the resource type and target type.

- **Endpoint:**

`DELETE /projects/{projectId}/customfields/{resourceType}/{targetType}`

- **Response:**

```bash
204 - No Content

```

#### Delete Global custom fields

To delete a custom field's configuration for a specified resource type and target type within a given **Global** context. This involves identifying the target scope, and specifying the resource type and target type.

- **Endpoint:**

`DELETE /customfields/{resourceType}/{targetType}`

- **Response:**

```bash
204 - No Content

```

## Artificial Intelligence (AI)

The AI endpoints provide access to Large Language Model (LLM) capabilities.

Their availability depends on the following system configuration and the user’s permissions:

- **System configuration:** The AI service must be enabled by a system administrator via the following system property: `com.siemens.polarion.copilot.enabled`
- **User license:** The user must have the Polarion Copilot Add-on license.
- **User permissions:** The user must have the permission to use Polarion Copilot (`com.polarion.copilot.use`) and the permission to use the Polarion Copilot API (`com.polarion.copilot.api.use`).

If AI is not enabled or the user is not allowed to access it, the endpoints return a `FORBIDDEN` response.

### Get available models

Retrieve LLM models currently available on the server.

- **Example Endpoint**

`GET /llmsGET /llms`

- **Response:**

```bash
{
  "links": {
    ...
  },
  "data": [
    {
      "type": "llms",
      "id": "basicModel",
      "attributes": {
        "name": "basicModel"
      }
    },
    {
      "type": "llms",
      "id": "reasoningModel",
      "attributes": {
        "name": "reasoningModel"
      }
    }
  ]
}

```

### Generate completion

Generate a completion using a configured LLM.

- **Example Endpoint**

`POST /llms/actions/generateCompletion`

- **Request Body:**

````bash
{
  "model": "basicModel",
  "messages": [
    {
      "role": "system",
      "content": "You are an expert in requirements engineering."
    },
    {
      "role": "user",
      "content": "Analyze the following requirement and return its type (functional or non-functional): ```The system shall encrypt all user data.```"
    }
  ],
  "responseFormat": {
    "type": "jsonSchema",
    "schema": {
      "type": "object",
      "properties": {
        "type": {
          "type": "string",
          "enum": [
            "functional",
            "non-functional"
          ]
        }
      },
      "required": [
        "type"
      ],
      "additionalProperties": false
    }
  }
}

````

- **Response:**

```bash
{
  "data": {
    "message": {
      "role": "assistant",
      "content": "{\"type\":\"functional\"}"
    }
  }
}

```

# Advanced Configuration

In addition to the `com.siemens.polarion.rest.enabled` and `com.siemens.polarion.rest.swaggerUi.enabled` properties that [enable REST API](enable-rest-api.html) and the [Swagger UI](enable-rest-api.html), the following additional properties let you further customize the Polarion REST API. They are added to the [polarion.properties](https://docs.sw.siemens.com/en-US/doc/230235217/PL20250606201928474.polarion_help_sc.xid1465510/xid1482566) file.

## com.siemens.polarion.rest.cors.allowedOrigins

This property enables [CORS](https://en.wikipedia.org/wiki/Cross-origin_resource_sharing) (Cross-Origin Resource Sharing) access to all REST endpoints.

### Supported Values

- Empty list CORS is disabled.
- `*` CORS requests from any host are allowed.
- `Comma separated list of URI` (schema://host:port) Only CORS requests from these origins are allowed.

(Standard ports - `80` for `HTTP`, `443` for `HTTPS` - should be omitted. For example, `https://secured.host.name` instead of `https://secured.host.name:443`)

Note

- Requests from the same origin that Polarion is running on are always allowed, regardless of the value in this property.
- Requests from origin set in `com.siemens.polarion.rest.baseUrl` are always allowed, regardless of the value of this property. (If not defined, `baseUrl` is used as the default value.)
- For a [Cluster](https://docs.sw.siemens.com/en-US/doc/230235217/PL20250606201928474.polarion_help_sc.xid1944826/xid1397882) deployment, we recommend that you set the property in the shared `polarion.properties` file on the cluster machine to ensure that all the nodes use the same setting.

## com.siemens.polarion.rest.maxPageSize

This property sets the limit for the maximum page size that can be requested by a REST API client.

- The default value is `100`.
- The absolute maximum page size is `10000`. (Setting this property to a number larger than `10000` has no effect.)

Note

For a [Cluster](https://docs.sw.siemens.com/en-US/doc/230235217/PL20250606201928474.polarion_help_sc.xid1944826/xid1397882) deployment, we recommend that you set the property in the shared `polarion.properties` file on the cluster machine to ensure that all the nodes use the same setting.

## com.siemens.polarion.rest.defaultPageSize

This property defines the default number of returned items for REST API endpoints.

- If `defaultPageSize` exceeds the `maxPageSize` (see above), the `defaultPageSize` value is replaced by the `maxPageSize` value.
- The default value is `100`

## com.siemens.polarion.rest.bodySizeLimit

This property limits the size of the request body in an HTTP request.

The limit is only valid for `HTTP` `POST`, `HTTP` `PATCH`, and `HTTP DELETE` requests with `application/json` content type.

The value is given as bytes.

Note

This limit also applies to the size of the JSON file that is used when POSTing or PATCHing attachments. Also, using a high value for this property might make the Polarion server vulnerable from a security point of view.

## com.siemens.polarion.rest.maxIncludedSize

This property specifies the number of results that are returned per page in the **Included** section for REST API endpoints. For more information, see [Limit of included resources](basic-concepts.html#limit-of-included-resources).

- If the number of entities exceeds the `maxIncludedSize`, they are not added to the **Included** section when requested.
- The default value of this property is `500` items, with a maximum limit of `10 000`.
- If the provided value exceeds the `10 000` limit, it is capped.

## com.siemens.polarion.rest.maxRelationshipSize

This property specifies the maximum number of entities that can be returned for each of the resource(s) relationship field(s). For more information, see [Limit of included resources](basic-concepts.html#limit-of-included-resources).

- If the number of entities exceeds the `maxIncludedSize`, they are not added to the **Included** section when requested.
- The default value of this property is `500` items, with a maximum limit of `10 000`.
- If the provided value exceeds the `10 000` limit, it is capped.

# Authentication

The Polarion REST API supports authentication via Personal Access Token (PAT), JSON Web Token (JWT), and Teamcenter SSO (TCSSO) token.

Every call is individually authenticated, there are no persistent sessions.

- [Use Personal Access Tokens (PATs)](authentication.html#authentication-with-pat).
- [Use the Polarion portal](authentication.html#authentication-from-polarion-without-pat).
- [Use JSON Web Token](authentication.html#authentication-via-json-web-token-jwt).
- [Use Teamcenter SSO (TCSSO) token](authentication.html#authentication-via-teamcenter-sso-tcsso-token).

Tip

Personal Access Token-based authentication must be generated/renewed by logging into Polarion. The non-PAT authentication methods (JWT and TCSSO) are more suitable for integrations.

## Authentication with PAT

When you try to access Polarion's REST API you must provide a Polarion Personal Access Token in the Authorization header of each request.

`Authorization: Bearer {personal_access_token}`

Example

`Authorization: Bearer 432ewrgdtfhdtdr54ztrhdfjfg`

If the token is missing or invalid, a `401 Unauthorized` response is returned.

Tip

The API reference documentation in [Polarion's SDK](https://docs.sw.siemens.com/en-US/doc/230235217/PL20251205943446859.scripting_guide) contains examples on how to provide an authentication token using the command line or different scripting languages.

### Test REST Endpoints using the Swagger UI

To test the Polarion REST endpoints using the Swagger UI, click **Authorize** and authorize yourself using your Polarion Personal Access Token.

(Personal Access Tokens are already used for SOAP Web Services.)

- Any Polarion user can [create and manage their own tokens](https://docs.sw.siemens.com/en-US/doc/230235217/PL20251205943446859.polarion_help_sc.xid1465510/xid2210302).

- Polarion administrators can [immediately revoke another user's tokens](https://docs.sw.siemens.com/en-US/doc/230235217/PL20251205943446859.polarion_help_sc.xid1465510/xid2210303) in an emergency.

- Personal Access Tokens can also be renewed.

  - Polarion users can renew their own (expired or valid) tokens.
  - Privileged users (with `MANAGE USER` permissions in the *Global* context) can also [renew](https://docs.sw.siemens.com/en-US/doc/230235217/PL20251205943446859.polarion_help_sc.xid1465510/xid2210304#hkg1683231443766__section_e5b_2zf_q1c) the expired/valid tokens of other users.
  - Administrators can set the number of days before a renewed token expires with the following property in the [polarion.properties](https://docs.sw.siemens.com/en-US/doc/230235217/PL20251205943446859.polarion_help_sc.xid1465510/xid1482566) file: `com.siemens.polarion.security.personalAccessToken.maxDaysBeforeRenewedTokenExpiry` (The default value is `90` days.)

Tip

See [Access Token support](https://docs.sw.siemens.com/en-US/doc/230235217/PL20251205943446859.polarion_help_sc.xid1465510/xid2020139) in Polarion's Help for more information.

## Renew PAT via API

You can renew Personal Access Tokens through the `com.polarion.platform.security.accesstoken.IUserAccessTokenService.renewToken(String, String)` Java API.

**Example of a custom script executed via a job to renew a Personal Access Token:**

```bash


```

Tip

See [Configure the Scheduler](https://docs.sw.siemens.com/en-US/doc/230235217/PL20251205943446859.polarion_help_sc.xid1465510/xid1552819) in Administrator and User Help to learn how to define and schedule a **Job**.

## Authentication from Polarion without PAT

To access the Polarion REST API, you can issue an `HTTP` request using JavaScript while within an active Polarion session without having to generate a Personal Access Token (PAT).

To authenticate through the Polarion REST API, you can use the `X-Polarion-Rest-Token` header, which is bound to the authenticated session and follows the same lifecycle.

The generated `X-Polarion-REST-Token` is only valid for the current session and becomes invalid when you log out.

Tip

This feature is useful in **Report Page** [widgets](https://docs.sw.siemens.com/en-US/doc/230235217/PL20251205943446859.polarion_help_sc.xid1465510/xid1483791), where you can fetch data on demand.

To use the `X-Polarion-REST-Token`, add the following property to the [polarion.properties](https://docs.sw.siemens.com/en-US/doc/230235217/PL20251205943446859.polarion_help_sc.xid1465510/xid1482566) file:

`com.siemens.polarion.rest.security.restApiToken.enabled=true`

Warning

- The Polarion REST API using the `X-Polarion-REST-Token` can perform read/write operations on behalf of the user viewing a Polarion **Report Page**.
- To prevent malicious code from running secretly, administrators should ensure that `WRITE` **Page** permissions are defined correctly and are only enabled for trusted users.

To authenticate your Polarion REST API call, supply the `X-Polarion-REST-Token` header in each Polarion REST API request.

The value of the header is obtained from the `top.getRestApiToken()` function, which generates the session-bound token.

The function is available in the Polarion UI and can be called directly using `top.getRestApiToken()`, `window.getRestApiToken()`.

The `getRestApiToken()` function is available on:

- **Live Report Pages** (In the scripting widgets like "**Script - Block**".)
- Velocity and Java **Widgets**
- **Classic Wiki Pages**
- **LiveDoc Documents** (In the Wiki Content sections.)
- Any other place with direct access to Polarion's UI DOM where JavaScript can be executed.

The `getRestApiToken()` function is **NOT** available on:

- **Work Item/Document Sidebars** (JavaScript execution support is not yet implemented.)
- Workflow Scripts (Since there is no UI with a DOM to access the function.)
- Jobs (same as Scripts),
- Any other place where the Polarion UI is not directly accessible.

**Here's an example of how to authenticate using the X-Polarion-REST-Token token:**

```bash
 

```

The following example shows how to use the Polarion REST API to create a new test **Work Item** using the `X-Polarion-REST-Token` token.

When the script is used on a **Classic Wiki Page** (or in the Wiki Content section of a **LiveDoc Document**), you must apply special handling via `{pre}` tags.

**If not, Polarion processes the xWiki syntax, which changes and breaks the JavaScript code.**

```bash
## This script creates a new Work Item in the current project from Classic Wiki Page, 
Document page, LiveReport Pages & Work Item Velocity/Java Form Extensions   

{pre} 
 
#set($projectId = "") 
#if($document.getProjectId() != "") 
#set($projectId = $document.getProjectId()) 
#elseif ($page.getProject() != "") 
#set($projectId = $page.getProject()) 
#elseif ($page.getReference().projectId() != "") 
#set($projectId = $page.getReference().projectId()) 
#elseif ($object.getContextId().getContextName() != "") 
#set($projectId = $object.getContextId().getContextName()) 
#end   


{/pre}

```

## Authentication via JSON Web Token (JWT)

Polarion supports JSON Web Token (JWT) authentication for its REST API, offering flexibility and enhanced security by allowing configuration with various Identity Providers (IDPs) and supporting both symmetric and asymmetric signing methods, with detailed setup instructions provided for generating and processing JWTs.

JSON Web Token (JWT) is a widely used mechanism for authentication and authorization. Polarion supports JWT as an authentication mechanism for the REST API layer in addition to the PAT. You can configure Polarion to accept JWTs generated by various Identity Providers (IDPs). This offers more flexibility compared to PAT, especially for integrations.

In this section we discuss the following JWT authentication-related topics:

- Generating a JWT
- Configuring the authenticator to process JWT

### Generating a JWT

JWTs are generated by various IDPs, for example Google, Facebook, Twitter, Keycloak, and so on. The structure of the JWT has to have information about the Polarion user who would be executing Polarion functionalities when authenticated with the supplied token.

The following configuration is needed to process the supplied JWT. This configuration is made as a JWT authenticator element in the `authentication.xml` file.

**Sample JWT structure:**

```bash
Header:
{
  "alg": "RS256",
  "kid": "key-1",
  "typ": "JWT"
}

Payload:
{
  "iss": "https://accounts.google.com",
  "sub": "admin",
  "name": "bharat",
  "role": "admin",
  "iat": 1744887113,
  "exp": 1744894313
}

Signature: JWT signature

```

The above JWT has an issuer (`"https://accounts.google.com"`) and a custom attribute (`"role" : "admin"`), which could be used to supply user information to the Polarion REST API.

**Authenticator configuration:**

```bash

    
    
        idp-with-jwks
        
            https://polarion-idp.com/jwks
            
                
                RS256
            
        
        
            $.role
            username
        
        jsonpath
    


```

The JWT authenticator (`jwt`) must have an issuer, for example `issuer`. The applicable JWT authenticator for a given JWT is selected by matching the issuer on the token with that of the `iss` element on the JWT authenticator. The given JWT can be verified in two ways:

- Retrieving the relevant JSON Web Key Set (JWKS) from the configured `jwks` URL under the `jwks` element.
- Using a local key, either HS256 (symmetric signing) or RS256 (asymmetric signing), which can be configured in the `key` element.

Both HS256 and RS256 algorithms are supported and have to be configured in the `algo` element. Due to shared key vulnerabilities, it is always suggested to use RS256 (asymmetric signing). The JWT configuration also provides an additional layer of security when the key is configured in the `authentication.xml` file through storage of the key in the User Account Vault. You can configure the JWT to store the key in the user account vault via specifying the `userAccountVaultKey` in the `key` element instead of the `key`. The `usage` attribute on the JWT authenticator specifies that the authenticator is to be used for the REST API. For the JWT authenticator, this is the only supported use case.

**Example:**

``

Here the actual key is in the user account vault with the following key: `"mySecretKey"`.

The user information specified in the supplied JWT is retrieved by making use of the `expression` specified in the `id` element under the `mapping` element of the relevant JWT authenticator. The information (content) specified under `id` has to be parsed by the specified `responseParser`. The `responseParser` can be `jsonpath` (the default) or `attr`.

**Example:**

```bash

     $.role
     username


```

The example implies that the `userid` to be used for the Polarion REST API calls is found by retrieving the `role` element in the JSON payload, which is specified as `admin`. The specified user must exist in Polarion for the user to be authenticated.

The JWT authentication attempts are logged in the log file with the `"JWT: AUTH :"` prefix. Each unique issuer can only be associated with one JWT authenticator, although multiple authenticators can be configured across different issuers. The REST API call must include the `X-Polarion-token-type` custom header with `jwt` value. The token must be supplied as an Authorization Bearer token.

## Authentication via Teamcenter SSO (TCSSO) token

Polarion's REST API supports Teamcenter SSO (TCSSO) tokens for authentication, requiring configuration of the `authentication.xml` file with a TCSS authenticator element and specific custom headers in API calls, while ensuring that the user IDs exist in both Polarion and Teamcenter systems for secure interaction.

Teamcenter SSO (TCSSO) tokens are supported by the Polarion REST API authentication. To use TCSSO-based authentication, you must configure the `authentication.xml` file by adding a `tcss` authenticator element, in addition to the usual `tcss` configuration steps.

When using TCSSO authentication for the REST API, the API call must include the following custom headers:

- `X-Polarion-token-type` with `tcss` value and
- `X-Polarion-user-id` with the required user-id as value.

The token must be supplied as an Authorization Bearer token.

The user with the supplied `user-id` in the header is used to interact with Polarion, and so this user must exist in the Polarion system. Adequate security measures have to be taken by the Teamcenter administrators to restrict the users who can interact with Polarion, as the same `user-id` needs to exist in both systems in the absence of a mapped user mechanism for generating TCSSO tokens by Teamcenter Security Services.

The tcss authentication attempts are logged in the log file with a `"TCSS: AUTH :"` prefix.

**Sample TCSS authenticator configuration:**

```bash


    
        http://tcss-server/login/sa
        http://tcss-server/identity
    


```

# Authorization

REST operations require two sets of permissions:

- Permission to call the endpoints. (REST API Endpoint permissions.)
- Permission to perform specific operations on items. (Read, Create, Update, Delete.)

## REST API Endpoint permissions

For REST API Endpoint permissions, the endpoints are categorized into:

- **Global Endpoints**: Endpoints where the `project` is not part of the path.

**Example:** `PATCH /users/{userId}, GET /jobs/{jobId}`

- **Project Endpoints**: Endpoints where the `project` is part of the path.

**Example:** `GET /projects/{projectId}/workitems/{workItemId}/approvals`

There are four REST API Endpoint permissions based on HTTP verbs:

`GET`, `PATCH`, `POST`, and `DELETE`.

For project-level endpoints, project-specific permissions can be set.

These permissions are set to `true` for the `admin` and `project_admin` roles in the **Global** context.

(And for all **Projects** by default.)

For all other roles, REST API Endpoint permissions are denied by default.

When an endpoint is called, REST API Endpoint permissions are checked.

To access `GET /users/{userId}`, `GET` permissions must be available for the calling user.

If the permissions are denied, the following error response is shown:

```bash
{

  "errors": [

    {

      "status": "403",

      "title": "Forbidden",

      "detail": "Sorry, you do not have the necessary permissions to perform this operation. 
      Please contact your Administrator if you need additional permissions.",

      "source": null

    }

  ]

}

```

### To allow REST API access for all users for both the Global and Project contexts

(Should be applied when you do not want to restrict REST API access at all.)

Enable all REST API Endpoint permissions (`GET`, `PATCH`, `POST`, `DELETE`) for the **everyone** Role in the **Global** (Default repository) context.

### Allow default REST API project-level access using project roles: (More restrictive)

Enable the following REST API Endpoint permissions in the global context:

- `GET`: **project_user**
- `GET`, `PATCH`, `POST`, `DELETE`: **project_assignable**, **project_approver**

It's important to remember that Polarion's REST API can be used externally by Polarion plugins and integrations and internally on **LiveReport Pages**.

Tip

- Restricting access to the REST API (even partially) can render REST API-based functionalities inoperable for some users.
- Administrators should first map out all the instances where the REST API is used and craft their REST API restrictions accordingly to avoid losing functionality.

**Example that illustrates REST API Endpoint permissions and Polarion item level permissions:**

**Projects**: `project_A`, `project_B`

**User**: `user_A`

**Roles**:

- `project_admin` in `project_A` (`project_admin` has `GET/POST/PATCH/DELETE` permissions)
- custom role with `GET` permissions in the **Global** context.
- `project_user` role in `project_B`. (`project_user` does not have any REST endpoint permissions.)

**Endpoints:**

1. `GET/all/workitems`
1. `GET/projects/project_A/workitems`
1. `/projects/project_B/workitems`

**user_A:**

- Can call endpoint (1) due to the custom role with the `GET` permission for the Global context.
- Can call endpoint (2) due to the `project_admin` role with `GET/POST/PATCH/DELETE` permissions for the Global context and these permissions are inherited (and not overridden at the project level) for `project A`.
- Can **NOT** call endpoint (3) because the `project_user` role has no required permission in the **Project** context.

Once the endpoint is called, Polarion permissions (Item-level) are checked, and a response is received based on the available permissions.

## Polarion Resource-Based-Permissions (Item-level)

Most operations in this API require permissions. Polarion permissions are fully applied, so the calling user must have the necessary permissions for an operation to use it.

If the available permissions are insufficient, the `403` response is returned.

Example

**The user does not have the required permissions to read project A:**

- **Example Request:**

  `GET /polarion/rest/v1/projects/A`

- **Response Body:**

```bash
{
  "errors": [
    {
      "status": "403",
      "title": "Forbidden",
      "detail": "You do not have permission to view Project 'A'."
    }
  ]
}

```

# Basic Concepts

## Requests and responses

### Requests

Client requests can contain a body unless they are `GET` requests. The format of the body should be as follows:

- `application/json`
- `multipart/form-data` (For requests that manage attachments. See the [Attachments](advanced-concepts.html#attachments) section for details.)

### Responses

Server responses may contain a body unless they are a `PATCH` request.

The body's format should be as follows:

- `application/json`

- `application/octet-stream`

  (For requests that manage attachment content. See the [Attachments](advanced-concepts.html#attachments) section for details.)

If there's a communication issue (for example, the Polarion server is down), the client receives a response with a body in `text/html` format.

Tip

The JSON schemas for request and response bodies are found in the [SDK documentation](sdk.html#default-sdk-location).

### Request headers

You must set up client request headers correctly.

The **Accept** header below indicates the expected format for server responses.

If set incorrectly, Polarion returns the following error:

```bash
{    
    "errors": [        
         {            
            "status": "406",             
            "title": "Not Acceptable"
        } 
    ]
} 

```

The **Accept** header is optional and can be omitted.

You can also set them to a wildcard value (`*/*`) or a concrete format based on the request:

`-H "accept: */*"`

`-H "accept: application/json"`

The **Content-Type** header indicates the body format of the request being sent. If set incorrectly, the following error response is returned:

```bash
{     
    "errors": [         
        {             
            "status": "415",             
            "title": "Unsupported Media Type"         
        }     
    ] 
}

```

The **Content-Type** header is more strict and **cannot** be omitted or set to a wildcard `*/*`.

The **Content-Type** header must match the format of the request body:

`-H "Content-type: application/json"`

### Typical responses

The following are typical responses you receive while making requests to Polarion's REST API.

- **200 OK**: For `GET` requests.
- **201 CREATED**: For `POST` requests creating a new object/resource. The resource identification is returned.
- **204 NO CONTENT**: For successful `POST` or `PATCH` requests that did not result in creating new resources. The updated content is not returned to save bandwidth.
- **400 BAD REQUEST**: With a detailed message if the request is malformed or otherwise invalid.
- **401 UNAUTHORIZED**: When the user it not authenticated.
- **403 FORBIDDEN**: When the authenticated user does not have sufficient permissions for the request.
- **404 NOT FOUND**: When the request contains a non-existing parameter and/or **Request Body**.
- **406 NOT ACCEPTABLE**: When the requested `Accept` header does not match the server's criteria.
- **409 CONFLICT_CODE**: When the `POST` **Request Body** contains an existing `ID`.
- **415 UNSUPPORTED MEDIA TYPE**: When the request contains an unsupported media format.

## Resource types and identification

Every resource provided by the **Polarion REST API** is uniquely identified by a pair of strings: `type` and `id`.

Where a particular revision of a resource needs to be referenced, these two strings are accompanied by a third one, the `revision`. (Not all resources support revisions.)

Use the Resource IDs as opaque values: they are not expected to be parsed, sliced, or combined by clients. Use them for example as keys in maps, or to match references in relationship with the included resources and so on. In most cases, instead of creating an endpoint URL (for example to fetch resource data) the clients should use links returned by the REST API endpoints (see [Links](basic-concepts.html#links)).

Note

`id`, `type`, or `revision` may also appear as attributes on certain resources. In that case, they are actual fields defined for that particular resource (for example `id` and `type` of **Work Items**, see the example below). While the `id` attribute typically has the same or a similar value as the `Resource ID`, it is important to understand that these attribute members are different from the REST API resource identification and are not interchangeable with them or directly related to them.

### Resource identification in the primary Data, ID and Type attributes

Example

`GET /polarion/rest/v1/projects/myProject/workitems/WI-123`

```bash
{
  "links": {...},
  "data": {
    "type": "workitems",
    "id": "myProject/WI-123",
    "attributes": {
      "id": "WI-123",
      "type": "requirement"
    },
    "relationships": {...}
    },
    "links": {...}
  }
}

```

### Resource identification in the relationship value (frozen Work Item link)

Example

`GET /polarion/rest/v1/projects/myProject/workitems/WI-1/linkedworkitems/relates_to/myProject/WI-2?fields[linkedworkitems]=@all`

```bash
{
  "links": {...},
  "data": {
    "type": "linkedworkitems",
    "id": "myProject/WI-1/relates_to/myProject/WI-2",
    "attributes": {
      "role": "relates_to",
      "revision": "1234"
    },
    "relationships": {
      "workItem": {
        "data": {
          "type": "workitems",
          "id": "myProject/WI-2",
          "revision": "1234"
        }
      }
    },
    "links": {...}
  }
}

```

## Links

Where specified (in the endpoint schemas), Rest API responses can include a link member with one or more named links (URLs).

They can be helpful for clients to make further calls to the REST API that manipulate the data or to get additional information about the returned data.

**The following table contains individual link type details:**

`self`

Link to the REST API endpoint request that produced the response (present in the root links section) or to the resource endpoint.

(Present in the resource links.)

Example

`https://myhost/polarion/rest/v1/projects/myProject/workitems/WI-123?fields%5Bworkitems%5D=title` `https://myhost/polarion/rest/v1/projects/myProject/workitems/WI-123`

`related`

Link to the REST API endpoint that returns the content of a relationship, present in the relationship links in the primary resource.

(If the relationship has its own endpoint).

Example

`https://myhost/polarion/rest/v1/projects/myProject/workitems/WI-123/comments`

`content`

Link to the REST API endpoint that downloads the file content of an attachment, present in the resource links of attachment resources.

Example

`https://myhost/polarion/rest/v1/projects/myProject/workitems/WI-123/attachments/1-myattachment.png/content`

`portal`

Link to the Polarion portal, present in the resource links and the list of response links.

(Currently, it is only supported by the `workitems` resource.)

Example

`https://myhost/polarion/redirect/project/myProject/workitem?id=WI-123` `https://myhost/polarion/redirect/project/myProject/workitems?query=type%3Arequirement`

`first`

Pagination link to the first page, present in the resource links and in the list response links. For more information, see [Pagination and resource sets](basic-concepts.html#pagination-of-resource-sets).

Example

`https://myhost/polarion/rest/v1/projects/myProject/workitems?page%5Bnumber%5D=1`

`prev`

Pagination link to the previous page, present in the resource links and in the list response links. For more information, see [Pagination and resource sets](basic-concepts.html#pagination-of-resource-sets).

Example

`https://myhost/polarion/rest/v1/projects/myProject/workitems?page%5Bnumber%5D=2`

`next`

Pagination link to the next page, present in the resource links and in the list response links. For more information, see [Pagination and resource sets](basic-concepts.html#pagination-of-resource-sets).

Example

`https://myhost/polarion/rest/v1/projects/myProject/workitems?page%5Bnumber%5D=4`

`last`

Pagination link to the last page, present in the resource links and in the list response links. For more information, see [Pagination and resource sets](basic-concepts.html#pagination-of-resource-sets).

Example

`https://myhost/polarion/rest/v1/projects/myProject/workitems?page%5Bnumber%5D=5`

### Link Example

- **Example Request:**

  `GET /polarion/rest/v1/projects/myProject/workitems/WI-123?fields%5Bworkitems%5D=title`

- **Response Body:**

```bash
{
    "links": {
        "self": "https://myhost/polarion/rest/v1/projects/myProject/workitems/WI-123?fields%5Bworkitems%5D=title"
    },
    "data": {
        "type": "workitems",
        "id": "myProject/WI-123",
        "attributes": {
            "title": "My Title"
        },
        "links": {
            "self": "https://myhost/polarion/rest/v1/projects/myProject/workitems/WI-123",
            "portal": "https://myhost/polarion/redirect/project/myProject/workitem?id=WI-123"
        }
    }
}

```

Note

You should treat the links as opaque. (Do not parse, slice, or combine them.) Only the URL parameters will typically be adjusted by a client. (See [URI Structure](access-rest-api.html#uri-structure))

## Fields

The **Attributes** and **Relationships** REST API resources are commonly referred to as **Fields**. In REST API requests and responses they usually correspond with built-in and custom object fields in Polarion. Simple types and enumerations (for example, `string`, `date`, and `status`) are represented as **Attributes**, while references to other objects, including custom object enumeration fields (for example author, categories) are represented as **Relationships**. Custom fields are identified by their **IDs** without any particular prefix or distinction from the built-in fields, the same as in the Polarion UI.

Tip

You can find the list of available built-in fields under the **Rest API Documentation** link of the REST API index. The same list is also available in the Swagger UI, if it is enabled.

A field is only returned in a REST API response if it has a non-empty value on the particular resource. Therefore, strings fields with an empty value, boolean fields with a `false` value, and `null` object references in relationships do not appear in responses, even if they have been specifically requested. However, an empty relationship may be returned if it provides the endpoint link to access the relationship value (for example for adding comments or attachments).

### Sparse fieldsets

With most `GET` requests, it is possible to request a particular set of fields to be returned using the `fields[resourcetype]` URL parameter.

Example

- `GET /users?fields[users]=@basic,globalRoles`
- `GET /projects/myProject?fields[projects]=id,trackerPrefix`

The value of each `fields[resource_type]` can be one of the following:

- The URL parameter, a comma-separated list of field IDs (attributes or relationships).
- The `@basic` keyword to include the basic set of fields predefined for the respective resource type.
- A combination of the `@basic` keyword and additional comma-separated field IDs.
- The `@all` keyword to include all the available fields for each resource of the given type.
- The `empty` value, so that no resource fields are returned, only the resource identification and links.

The default set of fields, returned when the `fields[]` parameter for the given type is not present, may differ based on the context of the request. Requests returning a set of resources will typically return no fields by default, while individual resource result will use the `@basic` fieldset.

### Notes

Warning

Use the **fieldset** feature with caution, notably, do not use `@all` or otherwise request more fields than strictly necessary for your particular request. Overusing @all or requesting too many fields, especially for large result sets, may cause unnecessary load on the server, network, as well as the client processing the response.

When using [Includes](basic-concepts.html#includes), the `fields[]` parameters apply to all resources of the given type in the response. It is also possible to specify multiple `fields[]` parameters, for example: `GET /projects/myProject/workitems?include=author&fields[workitems]=@basic&fields[users]=name`

Sparse fields `fields[workitems]` apply to all types of work items (see the note in [Resource types and identification](basic-concepts.html#resource-types-and-identification) on the differences between `type` as a resource vs. `type` as a field).

If unknown fields are listed in the parameter value, then they are silently ignored. Therefore, it is possible, for example, to request to fetch custom fields that are not defined for all **Work Items** in a result set without the risk of making a bad request.

When using Swagger UI, there is an autogenerated example to show all the available parameters for `fields[]`, but most of these would not apply, unless you really include the resources of all those types. You would typically use `fields[resource_type]` for the types to be returned only by your particular request. Also, the Swagger UI uses a different format for the `fields` parameter input: it is encoded as `JSON`, but it eventually adds the same individual `fields[type]` parameters in the generated request URL.

## Relationships

Each resource can have a relationship to other resources. They are presented alongside attributes.

Resources contain fields that are either **Attributes** or **Relationships**.

**Relationships** let you represent the entire hierarchical structure of a primary resource.

### Get Work Item with assignees and comments

- **Example Request:**

  `GET https://myhost/polarion/rest/v1/projects/myProject/workitems/WI-100?fields[workitems]=assignee,comments`

- **Response Body:**

```bash
{
    "links": {...},
    "data": {
        "type": "workitems",
        "id": "myProject/WI-100",
        "relationships": {
            "assignee": {
                "data": [
                  {
                    "type": "users",
                    "id": "sDeveloper"
                }
              ]
            },
            "comments": {
                "data": [
                  {
                    "type": "workitem_comments",
                    "id": "myProject/WI-100/1
                  }         
                ],         
                  "links": {           
                    "related": "https: //myhost/polarion/rest/v1/projects/myProject/workitems/WI-100/comments"          
                }        
              }      
             },     
             "links": { ... }   
          } 
    }

```

You can update regular relationships like **Assignee** or **Categories** with a `PATCH` request to the primary resource or through a relationship endpoint.

(See [Update using PATCH](advanced-concepts.html#update-using-patch) for details.)

### Update a Work Item relationship through primary resource: Add "mTest" user as an Assignee

- **Example Request:**

  `PATCH https://myhost/polarion/rest/v1/projects/myProject/workitems/WI-100`

- **Request Body:**

```bash
{
    "data": {
        "type": "workitems",
        "id": "myProject/WI-100",
        "attributes": {...},
        "relationships": {
            "assignee": {
                "data": [
                  {
                    "type": "users",
                    "id": "sDeveloper"
                }, 
                {
                    "type": "users",
                    "id": "mTest"
                }
              ]
           }
        }
    }

```

### Update a Work Item relationship through a relationship endpoint: Add "mTest" user as an Assignee

- **Example Request:**

  `POST https://myhost/polarion/rest/v1/projects/myProject/workitems/WI-100/comments`

- **Request Body:**

```bash
{
   "data": [
         {
             "id":"mTest",
             "type":"users"
         }
     ]
 }

```

**Relationships** to dependent resources like **Comments** and linked **Work Items** can be manipulated via a dedicated endpoint like `https://myhost/polarion/rest/v1/projects/myProject/workitems/WI-100/comments` in the example above. (See additional examples in the [Linked Work Items](advanced-concepts.html#linked-work-items) section). You can quickly identify these relationships because they include a related link that must be used to add/remove/modify the ties in the relationship section of the GET response of the primary resource.

### Update a Work Item relationship: Add a new comment to the Work Item

- **Example Request:**

  `POST https://myhost/polarion/rest/v1/projects/myProject/workitems/WI-100/comments`

- **Request Body:**

```bash
{
    "data": [
    {
      "type": "workitem_comments",
      "attributes": {
          "resolved": false,
          "title": "Test title",
          "text": {
              "type": "text/html",
              "value": "This is a comment"
            }
        }
    }
  ]
}

```

- **Response Body:**

```bash
{
    "data": [
      {
        "type": "workitem_comments",
        "id": "myProject/WI-100/2", 
        "links": {
           "self": "https://myhost/polarion/rest/v1/projects/myProject/workitems/WI-100/comments/2"     
        }   
     }
   ]
}

```

### Adding and removing a relationship

You can create or delete a relationship (single or multiple) using the **Relationships** endpoint.

The **Request Body** and **Response Body** are the same for the `POST`, `PATCH`, and `DELETE` methods.

- **Example Request:**

  `POST https://hostname/polarion/rest/v1/projects/drivepilot/collections/1/relationships/upstreamCollections`

- **Request Body:**

```bash
{
    "data": [
        {
            "type": "collections",
            "id": "drivepilot/1",
            "revision": "15687"
        },
        {
            "type": "collections",
            "id": "drivepilot/2",
            "revision": "15584"
        },
        {
            "type": "collections",
            "id": "drivepilot/2",
            "revision": "15587"
        }
    ]
}

```

- **Response Body:**

```bash
204 - No Content

```

### Things to keep in mind

Project-level objects, like **Categories** or **Plans**, are always referred to by their fully qualified **ID** when used in REST API requests or results. (This differs from how they appear in Lucene queries, for example.)

- **Example Request:**

  `GET /projects/myProject/workitems?query=categories.id:interface&fields[worktiems]=categories`

- **Response Body:**

```bash
{
  "links": {...},
  "data": [
    {
     "type": "workitems",
     "id": "myProject/WI-123",
        "relationships": {
            "categories": {
                "data": [
                  {
                    "type": "categories",
                    "id": "myProject/interface"
                }
               ]
            }
        },
        "links": {...}
    }
  ]
}

```

Sending an unqualified **ID** (for example, **interface** instead of **myProject/interface**) as part of a `PATCH` or `POST` request will typically result in a **400 Bad Request** response about an invalid or unresolvable object reference.

## Includes

When fetching resource data from endpoints, you can request to return additional related resources in the response along with the primary data. For example, if requesting a **Work Item's** resource, you may also want to know the name of the project it belongs to. **Project** is another resource in the `Project` relationship. (You can even return resources that are related to the related resources.)

To define which related resources to return along with the primary data, you must send an `include` query parameter. The value is a comma-separated list of relationship paths. A relationship path is a dot-separated list of relationship names. The related resources specified by the include parameter get returned in the response as an array in the included member.

By default, no fields of the resources in the included array get returned. To have some fields returned, they must be requested using the fields query parameter for the appropriate resource type. (For example, `fields[project]=name`. See [Fields](basic-concepts.html#fields) and [Sparse field sets](basic-concepts.html#sparse-fieldsets) for details.)

### Get a project name for a Work Item

- **Example Request:**

  `GET /projects/myProject/workitems/WI-123?fields[projects]=name&include=project`

- **Response Body:**

```bash
{
    "links": {...},
    "data": {
        "type": "workitems",
        "id": "myProject/WI-123",
        ...
    },
    "included": [
      {
        "type": "projects",
        "id": "myProject",
        "attributes": {
            "name": "My Project"
        },
        "links": {...}
      }   
   ] 
}

```

### Get an Author name and a Project Lead name for a Work Item

- **Example Request:**

  `GET /projects/myProject/workitems/WI-123?fields[users]=name&include=project.lead,author`

- **Response Body:**

```bash
{
    "links": {...},
    "data": {
        "type": "workitems",
        "id": "myProject/WI-123",
        ...
    },
    "included": [
      {
        "type": "users",
        "id": "myProjectLead",
        "attributes": {
            "name": "My Project Lead"
        },
        "links": {...}
    },
    {
        "type": "users",
        "id": "myAuthor",
        "attributes": {
            "name": "My Author"
        },
        "links": {...}
    }
  ]
}

```

### Notes

- The relationships used in the include parameter do not affect which fields are requested. (For that, you must use the `fields` query parameter.) So in the second example above, the author relationship of the **Work Item** is not returned. (Only the author is returned in the `included` member.) \`
- When a relationship path is used in the `include` parameter, only the leaf resources are returned in the `included` member. So in the second example above, the project resource is not returned in the `included` member, only its lead user is returned. If the project also needs to be returned, adjust the query parameter to, for example, `include=project,project.lead`.
- If some relationship in a relationship path in the include parameter is not found, the path is ignored, so it does not cause an error response. This is because relationships may be derived from custom fields so is only present in some of the primary resources.
- When the user does not have permission to read some of the relationships in a relationship path, this path will be skipped, and a related error will be returned in the `meta.errors` member in the response.
- If more related resources are specified than a limit of `100`, only this limited number of resources is included in the response. The limit is configurable via the `com.siemens.polarion.rest.maxPageSize` property.

## Limited responses and pagination

To prevent a large number of resources being returned at once, the REST API implementation applies certain default limits.

The following sections describe how these limits are applied to various responses.

### Pagination of resource sets

You can retrieve resource result sets in manageable chunks using pagination. Pagination is available for those REST API endpoints that return potentially large sets of resources, like **Work Items**, projects, comments, and so on.

Currently, the default page size is set to `100` and you can configure it using the `com.siemens.polarion.rest.defaultPageSize` system property. You can also set a non-default page size by using the `page[size]` URL parameter. You can configure the upper limit of the client-requested page size using the `com.siemens.polarion.rest.maxPageSize` system property.

By default, the first page of the results is returned, and you can request other pages either by directly using the `page[number]` URL parameter, or by following the pagination links that come as part of each response. For example:

Example

```bash
"links": {

    "self": "https://myhost/polarion/rest/v1/projects?page[size]=10&page[number]=5",

    "first": "https://myhost/polarion/rest/v1/projects?page[size]=10&page[number]=1",

    "prev": "https://myhost/polarion/rest/v1/projects?page[size]=10&page[number]=4",

    "next": "https://myhost/polarion/rest/v1/projects?page[size]=10&page[number]=6",

    "last": "https://myhost/polarion/rest/v1/projects?page[size]=10&page[number]=9"

  }

```

If the result is longer than one page, the response also contains information about the total size, for example:

Example

```bash
"meta": {

    "totalCount": 198

  }

```

If all the requested resources fit on one page, then all resources are returned, the `meta.totalCount` information is not present, and only the relevant pagination links are included.

### Pagination stability

There is no session context when paginating, each request is handled independently. That means that if the result set has changed between the requests (for example a new **Work Item** has been created that matches the query), the pages may overlap or may skip a resource.

As a workaround, we recommend using the `revision URL` parameter to pinpoint a particular data revision for example when paginating, so as to list all comments or all attachments of a resource:

```bash
https://myhost/polarion/rest/v1/projects/myProject/workitems/WI-123/comments?revision=556&page[number]=2

```

We also recommend using the `sort URL` parameter for resources that support it. Unsorted query results are displayed in a random order, which means that different pages can return the same data, or that some data can be missing.

Note

Stable pagination for primary resources like **Work Items** or projects is currently not available. However, it will be enabled in future REST API releases when the support for querying in baselines is introduced.

### Limit of returned relationships

The list of related resources displayed in the **Relationships** section of each resource can potentially contain a lot of items. Their number is automatically reduced to a limit set by the `com.siemens.polarion.rest.maxRelationshipSize` system property. See [Advanced configuration](advanced-configuration.html#advanced-configuration) for additional details.

If the limit caused a truncation of the list of related resources, the respective relationship's section also includes information about the total size of the list, for example:

Example

```bash
"meta": {

    "totalCount": 1211

  }

```

If the relationship size does not exceed the limit, the `meta.totalCount` information is not present.

The relationships representing dependent resources (like comments or attachments) always contain a `link.related` member pointing to their endpoint. This supports pagination and can be used to retrieve the full set of related resources.

Other relationships (like assignees or categories) do not provide a possibility to work around the limit at the moment, but the default value should be enough for the typical use cases.

### Limit of included resources

When using the Include feature described in the [Includes](basic-concepts.html#includes) section, the set of resources returned in the **Included** section is cut if their number exceeds a limit. This limit is set by the `com.siemens.polarion.rest.maxIncludedSize` system property, see [Advanced configuration](advanced-configuration.html#advanced-configuration) for details.

If **Included** is truncated, the primary response code is not affected (it might still be `200 OK`), but the top-level meta section contains an error entry informing about the cut. For example:

Example

```bash
"meta": {

  "errors": [ {

    "status": 400,

    "title": "Bad Request",

    "detail": "The number of included resources was more than the configured limit of 500.",

    "source": {

      "parameter": "include"

    }

  } ]

}

```

Note

If any related resource was not part of the response due to the limitation described in [Limit of returned relationships](basic-concepts.html#limit-of-included-resources), it is not part of the included resources at all, and such omission is not indicated in the meta section.

## Querying

The `query` URL parameter is available for selected endpoints.

The `query` syntax is the same as in Polarion Java APIs: click **Convert to Text** to obtain the clear text form of the query criteria for the `query` URL parameter for the query pane of the respective object type in the Polarion UI.

Note

The relevant `GET` single `resource/GET` collection endpoints can retrieve the revision of a specific item or items in a particular revision.

### Query Work Items

You can query **Work Items** on two levels: The Project level and Global levels.

The parameters and the response syntax are the same in both cases:

`GET /projects/{projectId}/workitems` is the endpoint for project-level querying.

It only returns **Work Items** that match the query in the given project.

`GET /all/workitems` is the endpoint for global, repository-level querying.

It returns matching **Work Items** in any of the projects that the currently authenticated user has read access to.

In the following example, only those resources (**Work Items**) are returned in the response body that are matching the `query` criteria in the request URL (the URL parameter is `query=type:systemrequirement AND status:reviewed`).

- **Example Request:**

  `GET https://myhost/polarion/rest/v1/projects/myProject/workitems?query=type:systemrequirement AND status:reviewed`

- **Request Body:**

`Not applicable (N/A)`

- **Response Body:**

```bash
{
  "links": {
    "self": "https://myhost/polarion/rest/v1/projects/myProject/workitems?query=type%3Asystemrequirement%20AND%20status%3Areviewed",
    "portal": "https://myhost/polarion/redirect/project/myProject/workitems?query=type%3Asystemrequirement%20AND%20status%3Areviewed"
  },
  "data": [
    {
      "type": "workitems",
      "id": "myProject/WI-123",
      "links": {
        "self": "https://myhost/polarion/rest/v1/projects/myProject/workitems/WI-123",
        "portal": "https://myhost/polarion/redirect/project/myProject/workitem?id=WI-123"
      }
    }
  ]
}

```

### Query Projects

When querying Projects, the endpoint is `GET /projects`.

In the following example, only those resources (Projects) are returned in the response body that are matching the `query` criteria in the request URL (the URL parameter is `query=trackerPrefix:PREFIX AND name:myName_* AND start:[20210401 TO 20210630]`). To make all project-related fields visible in the response body, the `fields[projects]=@all` (sparse fields) URL parameter was also used.

- **Example Request:**

  `GET https://myhost/polarion/rest/v1/projects?fields[projects]=@all&query=trackerPrefix:PREFIX AND name:myName_* AND start:[20210401 TO 20210630]`

- **Request Body:**

`Not applicable (N/A)`

- **Response Body:**

```bash
{
  "links": {
    "self": "https://myhost/polarion/rest/v1/projects?fields%5Bprojects%5D=%40all&query=trackerPrefix%3APREFIX%20AND%20name%3AmyName_%2A%20AND%20start%3A%5B20210401%20TO%2020210630%5D"
  },
  "data": [
    {
      "type": "projects",
      "id": "myName_tfs_defect",
      "attributes": {
        "id": "myName_tfs_defect",
        "name": "myName_tfs_defect",
        "description": {
          "type": "text/plain",
          "value": "This text comes from the project's Description"
        },
        "active": true,
        "start": "2021-06-02",
        "trackerPrefix": "PREFIX"
      },
      "relationships": {
        "lead": {
          "data": {
            "type": "users",
            "id": "myName"
          }
        }
      },
      "links": {
        "self": "https://myhost/polarion/rest/v1/projects/zskm9k_tfs_defect"
      }
    }
  ]
}

```

## Sorting

The `SORT` URL parameter is available for selected endpoints.

The syntax is the same as in Polarion URL (the same parameter value can be seen in the portal link and in the request URL, but the parameter name is `sort` vs. `sorting`).

`sort=~[sort field]`: Resources are sorted in DESCENDING order.

`sort=[sort field]`: Resources are sorted in ASCENDING order.

`sort=~[sort field 1] [sort field 2]`: Resources are first sorted by the first criterion (descending), then by the second criterion (ascending).

The separator between two or more sort criteria is a SPACE.

Note

Not all field names support sorting, also, some index fields may have a different name then a field name.

### Sort Work Items

When sorting Work Items, the endpoint is `GET /projects/{projectId}/workitems`.

In the following example, the returned resources (Work Items) in the Response Body are sorted into descending order based on the date they were created (the URL parameter is `sort=~created`).

The most recent **Work Items** are listed at the top.

- **Example Request:**

  `https://myhost/polarion/rest/v1/projects/myProject/workitems?sort=~created`

- **Request Body:**

`Not applicable (N/A)`

- **Response Body:**

```bash
{
  "links": {
    "self": "https://myhost/polarion/rest/v1/projects/myProject/workitems?sort=~created",
    "portal": "https://myhost/polarion/redirect/project/myProject/workitems?sorting=~created"
  },
  "data": [
    {
      "type": "workitems",
      "id": "myProject/WI-123",
      "links": {
        "self": "https://myhost/polarion/rest/v1/projects/myProject/workitems/WI-123",
        "portal": "https://myhost/polarion/redirect/project/myProject/workitem?id=WI-123"
      }
    },
...
  ],
  "meta": {
    "totalCount": 20
  }
}

```

## Revisions

The `revision` URL parameter retrieves the `HEAD` or a specific `REVISION` of a resource. It is available for selected endpoints. (Generally single-resource endpoints and endpoints listing their child resources.)

For example, `workitems/id`, `workitems/id/comments`.

### Get a Document from a revision

**Description**: In this example, the resource (**Document**) returned in the response body represents its state for a specific revision (URL parameter `revision=15357`). To make specific Document fields visible in the response body, the URL parameter `fields[documents]=title,type,status` (sparse fields) was also used.

- **Example Request:**

  `GET https://myhost/polarion/rest/v1/projects/myProject/spaces/_default/documents/myLiveDoc?fields[documents]=title,type,status&revision=15357`

- **Request Body:**

`Not applicable (N/A)`

- **Response Body:**

```bash
{
    "links": {
        "self": 
        "https://myhost/polarion/rest/v1/projects/myProject/spaces/_default/documents/myLiveDoc?fields%5Bdocuments%5D=title%2Ctype%2Cstatus&revision=15357"
    },
    "data": {
        "type": "documents",
        "id": "myProject/_default/myLiveDoc",
        "revision": "15357",
        "attributes": {
            "title": "myLiveDoc",
            "type": "generic",
            "status": "published"
        },
        "links": {
            "self": 
            "https://myhost/polarion/rest/v1/projects/myProject/spaces/_default/documents/myLiveDoc?revision=15357"
        }
    }
}

```

### Get linked Work Items for a Work Item from a revision

- **Example Request:**

  `GET https://myhost/polarion/rest/v1/projects/myProject/workitems/WI-123/linkedworkitems?revision=15367`

- **Request Body:**

`Not applicable (N/A)`

- **Response Body:**

```bash
{
    "links": {
        "self": "https://myhost/polarion/rest/v1/projects/myProject/workitems/WI-123/linkedworkitems?revision=15367"
    },
    "data": [
      {
        "type": "linkedworkitems",
        "id": "myProject/WI-123/parent/myProject/WI-122",
        "revision": "15367",
        "links": {
            "self": "https://myhost/polarion/rest/v1/projects/myProject/workitems/WI-123/linkedworkitems/parent/myProject/WI-122?revision=15367"
        }
    }
  ]
}

```

# Contact

[Talk to our experts](https://polarion.plm.automation.siemens.com/company/contact) to learn more about **Polarion's API** and how to integrate **Polarion** into your toolchain. They'll be happy to help.

Tip

- Download the [SDK Scripting Guide](https://docs.sw.siemens.com/en-US/doc/230235217/PL20250606201928474.scripting_guide).
- Download [Example Scripts](https://docs.sw.siemens.com/en-US/doc/230235217/PL20241023686685479.sdk_scripting).
- Download the PDF of the **REST API User Guide** and view all other Polarion documentation on [Support Center](https://polarion.plm.automation.siemens.com/documentation/latest).
- Visit our [Community Portal](https://community.sw.siemens.com/s/topic/0TO4O000000MihxWAC/polarion) and ask our experts.
- [file a Support Ticket](https://support.sw.siemens.com/en-US/support-case/open) on **Support Center**.

# Enable REST API

Polarion's REST API is disabled by default, but administrators can enable it by setting the following property in the [polarion.properties](https://docs.sw.siemens.com/en-US/doc/230235217/PL20251205943446859.polarion_help_sc.xid1465510/xid1482566) file:

`com.siemens.polarion.rest.enabled=true`

## (Optional) Enable the Swagger UI

The [Swagger UI](https://swagger.io/tools/swagger-ui/) is generated automatically from Polarion's OpenAPI implementation.

When enabled, your development team or end consumers can view and interact with API resources without needing any implementation logic in place.

To enable Swagger UI, set the following property to true in the [polarion.properties](https://docs.sw.siemens.com/en-US/doc/230235217/PL20251205943446859.polarion_help_sc.xid1465510/xid1482566) file:

`com.siemens.polarion.rest.swaggerUi.enabled=true`

Note

- Both properties are system-level properties so you must set them in the `polarion.properties` file before starting Polarion.
- They are both disabled by default to give you more control over security.
- `com.siemens.polarion.rest.enabled` must be set to `true` for `com.siemens.polarion.rest.swaggerUi.enabled` to have an effect.

# Limitations and Known Issues

The following list identifies the most common use cases that are not yet available via the Polarion REST API:

## General

- Trigger a PDF export of objects that already support it via the UI.
- Apply default sorting to all collection GET requests to achieve consistency of returned paginated results.
- Introduce a user-friendly way to work with an object's history.
- Get the specific revision ID of an object requested in the `HEAD` revision.
- Support a variable identifying the current user (`@me`) in the REST calls.
- Introduce full support for the OpenAPI Generator.
- Support a way to exclude specific Sparse Fields from the `@all` definition in queries.
- Provide a generic Search that will return all Polarion Objects.
- Invoke REST API calls from Workflow scripts.
- Enable the use of eSignatures via the REST API.

## Administration

- The majority of the *Administration* topics are not yet available through the REST API.

- The exceptions (with varying levels of coverage) are:

  - Custom Fields,
  - Enumerations,
  - Icons,
  - Licenses,
  - Project Templates,
  - Projects,
  - Roles,
  - User Groups,
  - Users.

## Baselines

- The Polarion REST API has not yet covered this topic.
- Support Collection, Document, and Project Baselines.

## Build Management

- The Polarion REST API has not yet covered this topic.

## Categories

- The Polarion REST API has not yet covered this topic.

## Documents

- Assign Document Layout to each Work Item.
- Import Word Documents using an existing document import configuration.
- Derive a Document/update a derived Document.
- Use Word/ReqIF Round-trip.
- Move/rename a Document.
- Delete a Document.
- Delete a Document Attachment.
- Copy a Document with a different title.
- Override a branched Work Item in a Document.
- Move Work Items to the Recycle Bin.
- Work with Document Approvals.
- Update a derived Document.
- Compare Documents.
- Get the Wiki Content section as already processed HTML instead of just its code.
- Combine/append Documents.

## Enumerations

- The following out-of-the-box Object enumerations are not supported in the current version and are treated as String enumerations:

  - Builds,
  - Project Groups,
  - Time Points,
  - Wiki Pages.

- Management of the Dependent Enumerations.

## Icons

- Replace or delete an icon.

## Pages

- Download a Page as an archive.
- Import Page Archive into Polarion.

## Permissions

- Provide more granular per-endpoint permissions for Global- and Project-level endpoints.

## Plans

- Get Plan Statistics.

## Projects

- Get all users of a Project.

## Project Groups

- The Polarion REST API has not yet covered this topic.

## Spaces

- The Polarion REST API has not yet covered this topic.

## Test Management

- Support automatic Defect creation when importing Test Results.
- List recent Tes t Records for a given Work Item.
- Provide a stand-alone Execute Test Case action.
- Work with Testing/Test Step Table Configuration.

## Time Points

- The Polarion REST API has not yet covered this topic.

## User Management

- Get a list of all User Roles or User Groups.
- Delete a User.
- Manage User Roles or Groups.
- Provide REST API endpoints to manage Personal Access Tokens.

## Variant Management

- The Polarion REST API has not yet covered this topic.

## Wiki Pages

- Polarion REST APIs will not cover this topic because it is a feature that is being phased out.

## Work Items

- Add the option to add or remove a Work Item to/from a Plan.
- Provide an action to Duplicate a Work Item.
- Get Rich Text field content as processed HTML instead of its source code.
- Provide a way to trigger the Initial Workflow action when posting a new Work Item in a single request.
- Improve Work Records creation so it correctly adjusts the Time Spent and Remaining Estimate fields.

The Polarion REST API is actively being developed, and our main focus for future Polarion releases is to cover as many gaps identified in the above list as possible. This list is internally being maintained based on the feedback we receive from you, and it will be updated with every Polarion release.

Note

**Call to Action!**

If you identify a specific gap that is blocking you from implementing your use case with the Polarion REST API, no matter if it is on the above list or not, please [file a Support Ticket](https://support.sw.siemens.com/en-US/support-case/open) on [Support Center](https://support.sw.siemens.com/en-US/) and describe the use case and the gap. Our Support Engineers will analyze your request and track it in our system. Your reports greatly affect the overall priority we use to decide which gap will be covered next.

- [Introduction](overview.html)
- [Enable REST API](enable-rest-api.html)
- [Access REST API](access-rest-api.html)
- [Authentication](authentication.html)
- [Authorization](authorization.html)
- [Basic Concepts](basic-concepts.html)
- [Scenarios and Advanced Concepts](advanced-concepts.html)
- [Advanced Configuration](advanced-configuration.html)
- [Limitations and Known Issues](limitations-and-known-issues.html)
- [Polarion REST API](rest-api-spec.html)
- [Polarion SDK](sdk.html)
- [Contact](contact.html)

# Introduction

Welcome to the `Polarion REST API User Guide`. This document is intended for developers, designers, architects, and anyone planning to use Polarion's Representational State Transfer (REST) API implementation.

Tip

This is the `Polarion 2606` version of the `REST API User guide`. Older versions are on Support Center at:

- [Polarion 2512](https://docs.sw.siemens.com/en-US/doc/230235217/PL20250606201928474.polarion_help_sc.xid2134849/xid2134871)
- [Polarion 2506](https://docs.sw.siemens.com/en-US/doc/230235217/PL20241023686685479.polarion_help_sc.xid2134849/xid2134871)
- [Polarion 2410](https://docs.sw.siemens.com/en-US/doc/230235217/PL20240424963191224.polarion_help_sc.xid2134849/xid2134871)
- [Polarion 2404](https://docs.sw.siemens.com/en-US/doc/230235217/PL20231017526942799.polarion_help_sc.xid2134849/xid2134871)
- [Polarion 2310](https://docs.sw.siemens.com/en-US/doc/230235217/PL20230412292748000.polarion_help_sc.xid2134849/xid2134871)
- [Polarion 2304](https://docs.sw.siemens.com/en-US/doc/230235217/PL20221020258116340.xid2134849/xid2134871)
- [Polarion 22 R2](https://docs.sw.siemens.com/en-US/doc/230235217/PL20220419389880191.xid2134849/xid2134871)

The **Polarion REST API** gives external applications an integration layer with Polarion that gives you greater control over the information you use in both Polarion and the applications you use daily.

The **Polarion REST API** architecture follows Rest API community standards like the **JSON** and **HTTP** methods and the Open API Specification. (Formerly known as the Swagger Specification).

The **Polarion REST API** is based on **JSON:API** with some deviations (for example, bulk create, update, and delete) from the JSON:API specification.

(So while the **Polarion REST API** is very similar to a **JSON:API** it deviates slightly from the standard.)

Note

The **Polarion REST API** relies on the existing **Java API** to deliver functionality and has the same restrictions and behavior. So, client-side features exclusively available on the user interface, like read-only fields, are unavailable through the **REST API**.

## Additional resources

You can find these additional resources and documentation on your Polarion server in the embedded SDK documentation:

`https://[Your_Polarion_server]/polarion/sdk/index.html#restapi`

- REST API Reference
- REST API schema file (`polarionrest.json`)

(The API Reference and Swagger UI are both generated based on this file.)

## General security warning

The data returned as part of **Polarion REST API** responses are provided "as is", and they correspond to the actual content stored by Polarion users via Polarion User Interfaces (UIs) or APIs. The usual caution needs to be exercised and REST API clients need to sanitize any data received from the server before rendering it or using it further.

In particular, note that HTML values of rich text and other special purpose fields (like Work Item descriptions, **Document parts'** content, the `homePageContent` field value in **Documents**, as well as other fields in other types of Polarion objects) are the internal raw data and are not intended to be used as ready-to-render HTML documents or snippets. Polarion cannot guarantee that such HTML content will be free from potentially malicious user-submitted scripts.

Similarly, when retrieving the attachment content via one of the content endpoints, please keep in mind that the actual attachment content is returned "as is", corresponding to the original octet stream previously uploaded to Polarion via UI or API. Polarion does not perform any validation, sanitization, or malware/virus scan upon uploading or downloading attachment files.

# Polarion REST API

- [Download OpenAPI Specification](polarion-rest-apispec.json)

# Polarion SDK

Polarion's SDK documents the available APIs, the embedded database schema, supported scripting languages and engines, libraries, source code, and code examples for developers who want to develop custom extensions to Polarion's standard features and functionality.

Tip

Need a custom script for Workflow **Functions** and **Conditions**?

- Download the [SDK Scripting Guide](https://docs.sw.siemens.com/en-US/doc/230235217/PL20250606201928474.scripting_guide).
- Download [Example Scripts](https://docs.sw.siemens.com/en-US/doc/230235217/PL20241023686685479.sdk_scripting).
- Visit our [Community Portal](https://community.sw.siemens.com/s/topic/0TO4O000000MihxWAC/polarion) and ask our experts.

The SDK content is installed to the file system along with all other programs and data.

## Default SDK location

You can find Polarion's SDK content in the `$POLARION_HOME$/polarion/sdk` folder of your installation.

Documentation is located in the `doc` sub-folder.

The `[POLARION_HOME]/polarion/sdk/index.html` file provides links to all SDK resources. You can access the SDK index page via your web browser by appending `polarion/sdk/` to the URL of your Polarion server.

Example

`http://polarion.mydomain.com/polarion/sdk/`

You can also copy the SDK from the installed location to local computers or host it on a file server.

## FAQ

1. *Where can I go to get help with scripts and Polarion API?*

   **Answer**: The best place to ask questions and get answers from other Polarion users and experts is on the [Polarion Community](https://community.sw.siemens.com/s/topic/0TO4O000000MihxWAC/polarion?__hstc=2015854.8dc351a9de470b6f5d50628b6bc56240.1641906576279.1641906576279.1641906576279.1&%3B__hssc=2015854.1.1641906576279&%3B__hsfp=3765005474) website.

1. *Do we need to call .`save()` on the object (`workflowContext.getTarget())` from which the workflow execute in a **ScriptCondition**?*

   **Answer**: You should never call .`save()` on the object that executes the workflow in a **ScriptCondition**. (It is not a script type meant for modifying Polarion Data.)

1. *Do we need to call `.save()` on the object (`workflowContext.getTarget())` from which the workflow execute in a **ScriptFunction**?*

   **Answer**: You should never call the `.save()` on the object that executes the workflow in a **ScriptFunction** because the script is already executing when Polarion saves the base object. If your **ScriptFunction** modifies another object (for example, a **Work Item** *linked* to the current **Work Item** that the workflow executes on), then yes, `.save()` should be called on this other object.

1. *I have updated the **Scheduler** entry for my `script.job` but when I try to run it manually in the **Monitor**, it does not pick up my modifications.*

   **Answer**: If you are updating the **Scheduler** entry of your job, make sure to refresh the **Monitor** page to pick up the changes.

1. *My `script.job` is failing when executed automatically via the **Scheduler** but not when I run it manually.*

   **Answer**: This means that the job does not have the same access to the Polarion data when it is executed automatically by the **Scheduler** because it uses the `polarion` user. (That only has read-only access to the data.) To execute it successfully, you must use the `doAsUser` method. See the **"Running parts of a script.job as another user with doAsUser"** section in the [SDK Scripting Guide](https://docs.sw.siemens.com/en-US/doc/230235217/PL20250606201928474.scripting_guide) for details.

1. *When executing a Groovy script an error is generated when trying to retrieve the current Groovy version.*

   **Answer**: The error is due to the use of an unsupported `getVersion()` call, probably from a Groovy script that was written for v1.5.

   **Example**

   `org.codehaus.groovy.runtime.InvokerHelper.getVersion();`

   With Groovy 2.4.21, it should be written as:

   `GroovySystem.getVersion();`

1. *When executing a `script.job`, I am getting this type of error: `Method (...) cannot be executed from a job script`.*

   **Answer**: Starting with **Polarion 2304**, some methods from `com.polarion.platform.security.ISecurityService` are being deactivated by default. You can unblock methods for `script.job` that are blocked by default by defining them in the following property in the polarion.properties file:

   `com.polarion.scripting.job.allowedSecurityServiceMethods`

   For example, if you want to unblock `addGlobalRoleToUser()` and `addContextRoleToUser()`:

   `com.polarion.scripting.job.allowedSecurityServiceMethods=addGlobalRoleToUser,addContextRoleToUser`

   For more information on the property, consult the [Polarion System Configuration Properties Reference](https://docs.sw.siemens.com/en-US/doc/230235217/PL20250606201928474.config_props_ref) guide on **Support Center**.

1. *When executing a **ScriptFunction** or **ScriptCondition**, I am getting this type of error: `Method (...) cannot be executed from a script`.*

   **Answer**: Starting with **Polarion 2304**, some methods from `com.polarion.platform.security.ISecurityService` are being deactivated by default. You can unblock methods for **ScriptFunction** and **ScriptCondition** that are blocked by default by defining them in the following property in the polarion.properties file:

   `com.polarion.scripting.allowedSecurityServiceMethods`.

   For example, if you want to unblock `addGlobalRoleToUser()`and `addContextRoleToUser()`:

   `com.polarion.scripting.allowedSecurityServiceMethods=addGlobalRoleToUser,addContextRoleToUser`

   For more information on the property, consult the [Polarion System Configuration Properties Reference](https://docs.sw.siemens.com/en-US/doc/230235217/PL20250606201928474.config_props_ref) guide on **Support Center**.
# Product Information Hub documentation

# Changelog

This page displays a curated, chronological list of notable updates for the Product Information Webservice across versions. It helps developers easily identify key changes between each project iteration. Developers can specify the API version by using the `Api-Version` HTTP header in the request.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [v2-earlyaccess] | 2023-02-14

> Latest ✔️. Used if no version is specified in the `Api-Version` header.

### Changes

- Added an endpoint for delivery information
- Made response property names standard camelCase: (ProductNumber -> productNumber)
- Added improvements to the swagger documentation

## [v1-earlyaccess] | 2022-11-22

> Deprecated ❌

### Changes

- Added caching of products for faster response times
- Requests to the APIs base-url are now redirected to /api
- Fixed A bug where product names would be missing from the response

## [v0] | 2022-11-10

> Deprecated ❌

### Changes

- Added authentication via API keys
- Added performance improvements
- Made error messages made more comprehensible

# Contact

For any inquiries, whether organizational matters like the procurement of API Keys or technical questions about the Product Information API, please direct your communication to the following email: [lifecycle-webservices.industry@siemens.com](mailto:lifecycle-webservices.industry@siemens.com)

# FAQ

***Why do I need to email someone for API access?***

We are actively exploring more efficient methods for granting access to the Product Information API. The email-based process is a temporary measure to ensure access while we develop a superior solution.

# Getting Started

To begin using the **Product Information API**, follow these steps:

1. Contact our support team to purchase an API Key. You can find details on our [contact information page](contact.html).
1. Include your API Key in the `Authorization` header of your HTTP requests.

## Example requests

```bash
API_KEY=your-api-key

# Obsolescence information about product "1PH8350-7MK40-0AX0"
curl -X 'GET' \
  'https://product-information-hub.siemens.cloud/api/products/1PH8350-7MK40-0AX0/obsolescence' \
  -H 'accept: application/json' \
  -H 'Authorization: ${API_KEY}'

# Delivery information about product "1PH8350-7MK40-0AX0"
curl -X 'GET' \
  'https://product-information-hub.siemens.cloud/api/products/1PH8350-7MK40-0AX0/delivery' \
  -H 'accept: application/json' \
  -H 'Authorization: ${API_KEY}'

# Information about your API key (ownership, available credits, ...)
curl -X 'GET' \
  'https://product-information-hub.siemens.cloud/api/api-key-details' \
  -H 'accept: application/json' \
  -H 'Authorization: ${API_KEY}'

```

Note

For more details on how to use the API and understand the response, please see the [API Specification](rest-api-spec.html)

- [Overview](overview.html)
- [Getting Started](getting-started.html)
- [Rest API](rest-api-spec.html)
- [Changelog](changelog.html)
- [FAQ](faq.html)
- [Contact](contact.html)

# Overview

## Introduction

> Empowering Your CMMS with the Product Information API 🚀

Mitigate the risk of functional obsolescence and maximize plant availability with our cutting-edge Product Information API. Get concrete action recommendations and identify potential supply bottlenecks early on, ensuring seamless operations. Stay informed about innovative features in successor products, unlocking optimization opportunities for your business. Experience the ultimate solution for proactive management and continuous improvement.

## Value

- **Efficient Search Capabilities**: Simply search for products by their product number, and let our API provide you with crucial information to make informed decisions.
- **Availability Horizon Insights**: Stay ahead with real-time data on product availability horizons, ensuring you can plan effectively for your business needs.
- **Successor and Substitute Products**: Discover successor and substitute products that can seamlessly replace your current products in the future, allowing for smooth transitions and continuous operations.

Unlock the full potential of our Product Information API, and experience the convenience of an all-in-one solution for managing product lifecycles and securing your business's future success.

# Product Information API

API Specification using OpenAPI.

- [Download OpenAPI Specification](product-information-api.spec.yaml)
# Railigent X documentation

# Contact

Would you like to learn more about IoT and AI in railways and how [Railigent X](https://www.mobility.siemens.com/global/en/portfolio/digital-solutions-software/digital-services/railigent-x.html) can improve your rail asset management?

For more information about Railigent X, get in touch with our experts. They'll be happy to help you.

[Get in touch with us](https://www.mobility.siemens.com/global/en/general/contact.html)

- [Overview](overview.html)
- [Contact](contact.html)
- [API Catalog](/apis.html?tag=Railigent+X)
- \*/

# Digital transformation for sustainable mobility – with Railigent X

One key to advancing sustainable transportation is to digitally increase the capacity of rail systems – with less resources.

Railigent X is the suite of applications that supports this goal. It enables rail operators, maintainers, and asset owners to better understand data, analyze assets, and derive actions. Resulting in enhanced operation and maintenance for up to 100% system availability.

[Learn more about Railigent X](https://www.mobility.siemens.com/global/en/portfolio/rail/services/digital-services/railigent.html)

______________________________________________________________________

## Quick facts at a glance

Why is it time for Railigent X?

Railigent X helps speed up action on the journey to decarbonized mobility. It marks the next logical step that follows Railigent. As an open ecosystem with our customers and partners, it supports your efforts to:

- Get more from your rail system with less resources and energy
- Accelerate digital transformation that combines the real and the digital worlds
- Connect, analyze, and optimize rail assets as a whole

What tasks does Railigent X support?

Railigent X offers data-driven solutions and decision support for:

- Optimized operations
- Preventive and predictive maintenance
- Lifecycle management of assets and the entire system

Who benefits from Railigent X?

Railigent X supports:

- Rail operators
- Maintenance providers
- Asset owners, e.g., leasing companies for vehicle fleets

What if you already have your own applications in place?

Railigent X is scalable in several stages and is therefore open for any type of use:

- With Data as-a-Service, all relevant data generated in the rail assets is available to you in a structured form via standardized application programming interfaces (APIs). You can integrate them into your own application environment to get a comprehensive overview of vehicles and rail infrastructure.
- It’s also possible to use Insights as-a-Service. This includes AI-supported trend analyses and predictive models that are also open for integration into existing applications.
- Another option: Railigent X provides rail-specific applications via Software as-a-Service. Larger players can selectively add these services to interoperate with their own application environment on demand

How do Railigent X users profit from Siemens Xcelerator?

Siemens Xcelerator takes an easy, open, and flexible approach to solving challenges. It’s an open digital business platform that enables customers to accelerate their digital transformation – faster, easier, and at scale.

Railigent X shares the same mindset as Siemens Xcelerator. This facilitates co-creation and collaboration with – and between – our partners and customers.

[Learn more about Railigent X and Siemens Xcelerator](https://www.mobility.siemens.com/global/en/portfolio/rail/services/digital-services/railigent.html#SharedprinciplesconsistentbenefitsRailigentXandSiemensXcelerator)

______________________________________________________________________

## Where you’re coming from: Typical challenges in the rail industry

### Reduce maintenance costs

Railigent X offers applications and data services that support you with predictive maintenance. It allows maintenance costs to be reduced by up to 15%.

### Cut the costs of delay

By combining all relevant data from the entire rail system, Railigent X helps reduce the costs caused by delays by up to 40%.

### Enhance availability and reliability

Railigent X supports a holistic view of rolling stock and infrastructure – for optimized operations, up to 100% system availability, and up to 10% greater reliability.

### Increase capacity – digitally

Railigent X combines asset data, insights, prediction models, and recommendations – this adds up to better network utilization and up to 100% occupancy.

______________________________________________________________________

## What you get: Scalable solutions and the power of choice

Railigent X leverages state-of-the-art technologies for connectivity, data analytics, data-driven workflows, and decision support.

The power of choice is always yours! Based on your existing data and application environment, you can select from:

- Asset data as an input for your systems (Data as-a-Service)
- AI-supported component analyses and predictive models (Insights as-a-Service)
- End-to-end support deployed in AI-powered applications (Software-as-a-Service).

# Asset data takeout

The Railigent X Takeout component provides direct access to raw and preprocessed asset data via presigned URLs. The URLs enable secure access to download files containing the raw and preprocessed data in a transparent way.

## Why receive data via the Takeout component?

To inform about changes in asset data in a timely manner via API.

The API gives you a convenient way to receive live information and raw data from your assets in near real-time as it comes from the assets. The data is not processed, providing you with a direct view on the asset and full transparency.

**Your Benefits:**

- Transparent view into asset data
- Raw, unprocessed data for the big picture
- Employ your own data processing on raw data streams

## Deprecation Policy

In the course of product and feature development at SMO, the API may evolve. We want to ensure that customers receive the information they need to take advantage of these features, and to update their systems in time. Features from previous releases may become deprecated with a new feature release (“breaking change”). SMO will notify API customers ahead of the release date, and ensure six months of backward compatibility. Examples of breaking changes are:

- Changes to endpoint URIs
- Changes in the API´s request / response structure
- Changes in HTTP methods used

### Important Considerations

To ensure a stable and reliable experience for all users, please be aware of the following policies.

- Data Retention: The URLs for data download are available for a specific period. Be sure to retrieve the data you need within this retention window, as older data may not be available through the streaming interface.
- Service Usage Limits: The API is subject to usage limits to ensure fair use and system stability. These limits may apply to the number of consumers, subscriptions, or the frequency of data retrieval requests.

# EU Data Act API Changelog

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).

## [1.0.0] - 2026-04-07

### Added

- Initial Release of **EU Data Act API**
- Compliance with Xcelerator REST API Guidelines V2

# Railigent X EU Data Act REST API – API Specification

[Download OpenAPI Specification](eudataact-api-v1.yml)

# Navigation

- [Overview](EUDataAct_Takeout.html)
- [EU Data Act API Changelog](changelog.html)
- [EU Data Act API v1 Specification](eudataact-api-v1.html)

# Streaming Data REST API

The API provides direct, programmatic access to live data streams from assets in the Railigent X data lake.

This REST API is as a straightforward and powerful tool for building real-time monitoring and data analysis applications. It enables the integration of live asset data directly into existing systems or the creation of responsive solutions.

Railway operators, owners, and lessors receive real-time data from critical onboard systems, enabling continuous tracking and analytics of their rail vehicle operations.

## Why real-time streaming via an API?

Following the Data as a Service (DaaS) paradigm, this API gives you a convenient way to receive live information and raw data from your assets in near real-time. This machine-to-machine interface allows you to process continuous data streams directly within your own business applications, tailored to your specific needs.

Pre-processed operational data can be easily integrated into customer systems, empowering you to develop insights and optimize vehicle utilization and maintenance strategies.

We provide a robust, standardized streaming service for developers building real-time monitoring and data analysis solutions.

**Your Benefits:**

- Enables real-time monitoring of key vehicle data in customer systems
- Choose your own tools. Full flexibility in processing and displaying vehicle data
- Monitor operational behavior, detect outliers, fill data gaps

## Available Data Services Packages

The availability and scope of data service packages are determined by contractual agreements and are specific to vehicle types.

Generally, the following data service packages are available:

Note: As a `Vectron` customer, all your agreed-upon data is accessible independently of specific service data packages.

Live Vehicle Information

The Live Vehicle Information package provides railway operators and leasing companies with a seamless way to integrate real-time data on the position, speed, and mileage of their rail vehicles directly into their IT systems. Unlike manual data collection or third-party hardware solutions, this API-based service delivers reliable, up-to-date information across all the vehicles in a customer’s fleet, with no additional effort required in workshops or on the vehicles. By easily incorporating critical vehicle data points, users can gain a continuous, near real-time overview of their fleet, enabling more informed decisions and quicker responses to changing conditions.

Key benefits of the Live Vehicle Information API include streamlined data integration, improved visibility and responsiveness, and hassle-free implementation. The standard package provides access to key asset data points: position, speed, mileage, vehicle ID, and trip ID, in near real-time. Customers can rely on consistent, accurate information across their entire fleet, regardless of hardware or software version, thanks to the platform's enterprise-grade reliability and security.

Occupancy Information

Occupancy Information is an optional service package for passenger vehicles. It provides the degree of occupancy per saloon, in percent.

This package is an optional add-on to Basic Vehicle Information.

Vehicle Status

The Vehicle Status package provides railway operators, owners, and lessors with real-time data from critical onboard systems, enabling continuous tracking and analytics of their rail vehicle operations. Delivered via the Railigent X Train API, this service offers visibility into how assets are being utilized, allowing users to optimize vehicle deployment and inform data-driven maintenance planning. Unlike installing third-party hardware, the API-based solution simplifies data access by providing pre-processed operational data that can be easily integrated into existing systems.

Key benefits of the Vehicle Status package include enhanced operational efficiency, improved vehicle utilization, and streamlined data integration. Railway operators and vehicle owners can analyze usage patterns to plan maintenance and circulation, while dispatchers and rental companies can leverage historical data to optimize fleet deployment. The flexible, scalable, and secure platform allows customers to seamlessly incorporate vehicle status information into their own analytics and reporting tools.

Subset of available data:

- Train Mode: Current operational mode of the train.
- Line Current: Electrical current in the train's power line.
- Line Voltage: Voltage in the train's power line.
- Tractive Effort: Force exerted by the train for movement.
- Cab: Status of the train's cab.
- Number of Locomotives: Count of locomotives in the train.
- Pantograph: Status of the train's pantograph.
- Wheel: Status of the train's wheels.
- Brake Effort: Force applied by the train's braking system.
- Train Length: Total length of the train.
- Train Shutdown: Indicates if the train is shut down.
- Train Weight: Total weight of the train.
- CCU1 Software Version: Software version of the first control unit.
- CCU2 Software Version: Software version of the second control unit.
- RDA Software Version: Software version of the remote data acquisition unit.
- Diesel Tank Liquid Level: Liquid level in the diesel tank.
- Main Air Pressure: Pressure in the main air system.
- Energy: Energy consumption or generation status.
- Pressure Brake Pipe: Pressure in the brake pipe.

Vehicle Counters

The Vehicle Counters package provides railway operators and leasing companies with a data service that delivers the current values of maintenance-relevant counters from their rail vehicles. This information allows users to plan data-driven maintenance by gaining visibility into the operation hours and switching cycles of critical components, rather than relying on time-based schedules. For dispatchers and rental companies, the historical counter data can also be leveraged to inform the efficient allocation and circulation of the vehicle fleet.

Key benefits of the Vehicle Counters package include optimized maintenance planning, enhanced fleet utilization. Unlike manual data collection or the installation of third-party hardware, the API-based solution provides accurate and up-to-date counter information with minimal effort required on the customer's part.

Subset of available data:

- Time in DC Mode 1500V On: Duration the vehicle operates in 1500V DC mode.
- Time in DC Mode 3000V On: Duration the vehicle operates in 3000V DC mode.
- Time in AC Mode 25KV On: Duration the vehicle operates in 25kV AC mode.
- Time in AC Mode 15KV On: Duration the vehicle operates in 15kV AC mode.
- Time Main Switch DC On: Duration the main switch is on in DC mode.
- Time Main Switch AC On: Duration the main switch is on in AC mode.
- Switch Cycles Main Switch DC: Number of cycles the main switch operates in DC mode.
- Switch Cycles Main Switch AC: Number of cycles the main switch operates in AC mode.
- Operating Time Vehicle: Total operating time of the vehicle.
- Time Speed Greater Than 2KMH: Duration the vehicle speed is greater than 2 km/h.
- Switch Cycles Main Air Compressor: Number of cycles the main air compressor operates.

Vehicle Config Data

The Vehicle Config Data Package provides rail operators with a centralized source of comprehensive vehicle data to support informed decision-making and efficient fleet management. The package documents the vehicle's key technical specifications, including maximum speed, starting traction, and braking force, which can be valuable for optimizing operations and maintenance.

The package covers essential vehicle master data, such as the UIC number, vehicle variant, and the latest software release available for that specific model. At the core of the package is a detailed list of the hardware and software versions currently installed on the vehicle's RDA (Remote Diagnostic Access) box. This information is complemented by a "Target/Current Analysis" (TCA) report, which identifies any deviations between the target software release and the versions currently running on the vehicle. This analysis helps pinpoint vehicles that may require software updates. For vehicles where Siemens is responsible for service, the package includes the target software release, enabling a direct comparison to the actual installed version.

Subset of available data:

- Vehicle master data: UIC number, vehicle variant. Latest software release available for this vehicle. For vehicles where SMO is responsible for service, we also provide the target release for each vehicle.
- Keeper, as known by Siemens. (Customers can request an update via help desk ticket.)
- Vehicle capabilities:
  - max speed
  - max starting traction
  - max braking force
- List of reported hard- and software versions from the RDA box: Actual vehicle configuration, as maintained
- Target / Current Analysis: actual software versions on vehicle vs current software versions on vehicle, indicating which device should receive which software, in which version.

## Deprecation Policy

In the course of product and feature development at SMO, the API may evolve. We want to ensure that customers receive the information they need to take advantage of these features, and to update their systems in time. Features from previous releases may become deprecated with a new feature release (“breaking change”). SMO will notify API customers ahead of the release date, and ensure six months of backward compatibility. Examples of breaking changes are:

- Changes to endpoint URIs
- Changes in the API´s request / response structure
- Changes in HTTP methods used

### Important Considerations

To ensure a stable and reliable experience for all users, please be aware of the following policies.

- Data Retention: Data within a stream is retained for a specific period. Be sure to retrieve the data you need within this retention window, as older data may not be available through the streaming interface.
- Service Usage Limits: The API is subject to usage limits to ensure fair use and system stability. These limits may apply to the number of consumers, subscriptions, or the frequency of data retrieval requests.

# Streaming Data REST API Changelog

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).

## [1.0.0] - 2026-04-07

### Added

- Initial Release of **Streaming Data REST API**
- Compliance with Xcelerator REST API Guidelines V2

# Navigation

- [Overview](StreamingAPI.html)
- [Streaming Data REST API Changelog](changelog.html)
- [Streaming Data REST API v1 Specification](streaming-api-v1.html)

# Railigent X Streaming Data REST API – API Specification

[Download OpenAPI Specification](streaming-api-v1.yml)

# Train API

The Train API allows you to retrieve near real-time information about rolling stock assets in your fleets. Railigent X uses cloud technology to normalize asset information across your fleets, and deliver it to you in a well-defined structure.

## Why Data services via an API?

Following the Data as a Service (DaaS) paradigm, Data services packages give you a convenient way of obtaining live vehicle data in near real-time via a machine-to-machine interface. You can then process this data in your own business applications, according to your own business needs.

We provide a growing number of standardized data services packages for vehicle owners and operators.

**Your Benefits:**

- Receive asset data in near-real time in order to optimize operations
- A clearly documented API makes it easy to integrate data into your systems
- Once set up, flexibly order additional data service packages

## Available Data Services Packages

Currently, you can order the following Data services packages:

Live Vehicle Information

The Live Vehicle Information package provides railway operators and leasing companies with a seamless way to integrate real-time data on the position, speed, and mileage of their rail vehicles directly into their IT systems. Unlike manual data collection or third-party hardware solutions, this API-based service delivers reliable, up-to-date information across all the vehicles in a customer’s fleet, with no additional effort required in workshops or on the vehicles. By easily incorporating critical vehicle data points, users can gain a continuous, near real-time overview of their fleet, enabling more informed decisions and quicker responses to changing conditions.

Key benefits of the Live Vehicle Information API include streamlined data integration, improved visibility and responsiveness, and hassle-free implementation. The standard package provides access to key asset data points: position, speed, mileage, vehicle ID, and trip ID, in near real-time. Customers can rely on consistent, accurate information across their entire fleet, regardless of hardware or software version, thanks to the platform's enterprise-grade reliability and security.

Designed for railway operators, locomotive owners, and leasing companies, the Live vehicle information package offers a seamless solution for integrating critical vehicle data into existing operational and fleet management systems.

Occupancy Information

Occupancy Information is an optional data package. For passenger vehicles, it provides the degree of occupancy of a wagon (or vehicle, in case the vehicle uses Jacobs bogies), in percent.

This package is an optional add-on to Basic Vehicle Information. It is aimed at operators of passenger vehicles.

Vehicle Status

The Vehicle Status package provides railway operators, owners, and lessors with real-time data from critical onboard systems, enabling continuous tracking and analytics of their rail vehicle operations. Delivered via the Railigent X Train API, this service offers visibility into how assets are being utilized, allowing users to optimize vehicle deployment and inform data-driven maintenance planning. Unlike installing third-party hardware, the API-based solution simplifies data access by providing pre-processed operational data that can be easily integrated into existing systems.

Key benefits of the Vehicle Status package include enhanced operational efficiency, improved vehicle utilization, and streamlined data integration. Railway operators and vehicle owners can analyze usage patterns to plan maintenance and circulation, while dispatchers and rental companies can leverage historical data to optimize fleet deployment. The flexible, scalable, and secure platform allows customers to seamlessly incorporate vehicle status information into their own analytics and reporting tools. Designed for railway operators, vehicle owners and lessors, as well as dispatchers and rental companies, the Vehicle Status package offers an enterprise-grade solution for accessing critical operational data, and supporting sophisticated analytics use cases.

Subset of available data:

- Train Mode: Current operational mode of the train.
- Line Current: Electrical current in the train's power line.
- Line Voltage: Voltage in the train's power line.
- Tractive Effort: Force exerted by the train for movement.
- Cab: Status of the train's cab.
- Number of Locomotives: Count of locomotives in the train.
- Pantograph: Status of the train's pantograph.
- Wheel: Status of the train's wheels.
- Brake Effort: Force applied by the train's braking system.
- Train Length: Total length of the train.
- Train Shutdown: Indicates if the train is shut down.
- Train Weight: Total weight of the train.
- CCU1 Software Version: Software version of the first control unit.
- CCU2 Software Version: Software version of the second control unit.
- RDA Software Version: Software version of the remote data acquisition unit.
- Diesel Tank Liquid Level: Liquid level in the diesel tank.
- Main Air Pressure: Pressure in the main air system.
- Energy: Energy consumption or generation status.
- Pressure Brake Pipe: Pressure in the brake pipe.

Vehicle Counters

The Vehicle Counters package provides railway operators and leasing companies with a data service that delivers the current values of maintenance-relevant counters from their rail vehicles. This information allows users to plan data-driven maintenance by gaining visibility into the operation hours and switching cycles of critical components, rather than relying on time-based schedules. For dispatchers and rental companies, the historical counter data can also be leveraged to inform the efficient allocation and circulation of the vehicle fleet.

Key benefits of the Vehicle Counters package include optimized maintenance planning, enhanced fleet utilization, and streamlined data access. Unlike manual data collection or the installation of third-party hardware, the API-based solution provides accurate and up-to-date counter information with minimal effort required on the customer's part. The comprehensive counter data can be easily integrated into existing maintenance planning, fleet management, and analytics systems. Designed for railway operators, locomotive owners, as well as dispatchers and rental companies, the Vehicle Counters package offers an enterprise-grade platform for accessing critical operational data.

Subset of available data:

- Time in DC Mode 1500V On: Duration the vehicle operates in 1500V DC mode.
- Time in DC Mode 3000V On: Duration the vehicle operates in 3000V DC mode.
- Time in AC Mode 25KV On: Duration the vehicle operates in 25kV AC mode.
- Time in AC Mode 15KV On: Duration the vehicle operates in 15kV AC mode.
- Time Main Switch DC On: Duration the main switch is on in DC mode.
- Time Main Switch AC On: Duration the main switch is on in AC mode.
- Switch Cycles Main Switch DC: Number of cycles the main switch operates in DC mode.
- Switch Cycles Main Switch AC: Number of cycles the main switch operates in AC mode.
- Operating Time Vehicle: Total operating time of the vehicle.
- Time Speed Greater Than 2KMH: Duration the vehicle speed is greater than 2 km/h.
- Switch Cycles Main Air Compressor: Number of cycles the main air compressor operates.

Vehicle Config Data

The Vehicle Config Data Package provides rail operators with a centralized source of comprehensive vehicle data to support informed decision-making and efficient fleet management. The package documents the vehicle's key technical specifications, including maximum speed, starting traction, and braking force, which can be valuable for optimizing operations and maintenance.

The package covers essential vehicle master data, such as the UIC number, vehicle variant, and the latest software release available for that specific model. At the core of the package is a detailed list of the hardware and software versions currently installed on the vehicle's RDA (Remote Diagnostic Access) box. This information is complemented by a "Target/Current Analysis" (TCA) report, which identifies any deviations between the target software release and the versions currently running on the vehicle. This analysis helps pinpoint vehicles that may require software updates. For vehicles where Siemens is responsible for service, the package includes the target software release, enabling a direct comparison to the actual installed version.

Subset of available data:

- Vehicle master data: UIC number, vehicle variant. Latest software release available for this vehicle. For vehicles where SMO is responsible for service, we also provide the target release for each vehicle.
- Keeper, as known by Siemens. (Customers can request an update via help desk ticket.)
- Vehicle capabilities:
  - max speed
  - max starting traction
  - max braking force
- List of reported hard- and software versions from the RDA box: Actual vehicle configuration, as maintained
- Target / Current Analysis: actual software versions on vehicle vs current software versions on vehicle, indicating which device should receive which software, in which version.

## Deprecation Policy

In the course of product and feature development at SMO, the API may evolve. We want to ensure that customers receive the information they need to take advantage of these features, and to update their systems in time. Features from previous releases may become deprecated with a new feature release (“breaking change”). SMO will notify API customers ahead of the release date, and ensure six months of backward compatibility. Examples of breaking changes are:

- Changes to endpoint URIs
- Changes in the API´s request / response structure
- Changes in HTTP methods used

# Train API Changelog

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).

## [1.1.0] - 2025-02-21

### Added

- Release of packages **Vehicle Counters**, **Vehicle Status** and **Vehicle Config Data**

### Changed

- Renamed package **Basic Vehicle Information** into **Live Vehicle Information**

## [1.0.0] - 2024-03-25

### Added

- Initial Release of **Basic Vehicle Information** and **Occupancy Information**
- Compliance with Xcelerator REST API Guidelines V2

# Navigation

- [Overview](index.html)
- [Train API Changelog](changelog.html)
- [Train API v1 Specification](train-api-v1.html)

# Railigent X Train API – API Specification

[Download OpenAPI Specification](train-api-v1.yml)
# RailXplore Fusion documentation

# Contact

For more information, feedback, and requests:

[Get in touch with us](https://www.mobility.siemens.com/global/en/general/contact.html)

# main navigation structure

- [Overview](overview.html)
- RailXplore Fusion
  - [Overview](RailXplore-Fusion-API/index.html)
  - [ESB API Specification](RailXplore-Fusion-API/railXplore-api.html)
- [Contact](contact.html)

# Digital Transformation for Sustainable Mobility with RailXplore Fusion

## Introduction

Integrating and automating workflows into day-to-day operations is key to harness the full potential of digitalization.

RailXplore Fusion enables the exchange of diagnostic data with various internal and external systems, applications, and data sources into a cohesive and seamless operational data hub concept

## How RailXplore Fusion works

Fusion provides a holistic API management, a powerful query engine and allows to define business logics to enable mission critical workflow automation and integration between our RailXplore diagnostic solutions and partner systems. Available APIs will be released via the Siemens Xcelerator Developer Portal and can be integrated project specific.

# RailXplore APM Enterprise Service Bus

The RailXplore APM Enterprise Service Bus allows you to retrieve near real-time information about your assets (devices) and their underlined generated events and alarms.

## Data services via the API

In line with the Data as a Service model, the data service package provides you with a convenient way of obtaining live device data in near real--time through a machine-to-machine interface. You can subsequently utilize this data in your own business applications tailored to meet your specific needs.

WWe offer an expanding range of standardized data service packages tailored for rail infrastructure owners, operators, and maintainers.

**Your Key Benefits:** The overall gains for your organization include the following key benefits such as:

- Maximizing Overall Performance
- Increased Asset Value
- Better Risk Management
- Improved Cost Effectiveness

and through the provided API, many concrete benefits are added to your systems such as:

- Receive asset (device) data and their underlined alarms in near-real time in order to enhance operational efficiency
- A well-documented API makes it easier to integrate relevant data and services into your systems
- Support for the API-service integration within your existing system
- Flexibly to order additional data service packages as needed

## Available Data Services

Currently, you can order the following Data services packages:

- RailXplore APM Enterprise Service Bus[1](#fn:1)

______________________________________________________________________

1. requires that your infrastructure is connected to RailXplore APM [↩](#fnref:1 "Jump back to footnote 1 in the text")

# Enterprise Service Bus Specification

## Introduction

TThe RailXplore APM (Asset Performance Monitoring) Enterprise Service Bus (ESB) allows you to retrieve near real-time information about your assets (devices) and their underlined generated events and alarms.

The main workflow includes three main steps, 1. subscribe, 2. get notified, 3. retrieve relevant information:

### RailXplore APM Interface via Enterprise Service Bus module

- RailXplore APM exposes several Interfaces for external systems enabling to consume data such as alarms, events, and device information.
- The main exposed interface is the Enterprise Service Bus (ESB)
- The interface publishes predefined alarms, events, and device details to the ESB and via RailXplore Fusion to its consumers.
- RailXplore Fusion can remove a subscribed consumer ( e.g. if consumer becomes unauthorized)
- RailXplore Fusion can block temporarily a subscribed consumer ( e.g. if there is uncertainty about a consumer and its behavior, preventing a denial-of-service attack )

### External Systems

- External systems can consume published data such as alarms, events, or device data once authorized.
- External systems are able to subscribe for particular events
- External systems are able to unsubscribe from particular events
- External systems are notified of an update for a subject
- The ESB sub-system can publish alarms, events, and device lists according to well defined formats

### API Properties and Capabilities

- the main API is based on the Observer design pattern
- one-to-many relationship: many external systems may connect to one publisher, the ESB, but can consume many topics
- to subscribe and unsubscribe a consumer
- authorization: shall be set as a precondition from the upper system
- public methods for data retrieval that can be called to retrieve specific data type for continuous pulling or pushing of data
- the required parameters and types public constants, variables, and naming scheme
- To consume data from RailXplore APM, the related asset types need to be connected and once contractually agreed, can be activated for subscription in RailXplore Fusion.

### Architectural Considerations

The following figure is a model for the fusion building block, illustrating the different cooperating layers (the producer and consumer of the events, alarms, and device list), and the related abstractions (generalization, such as topics, devices) and their underlined specializations (Concretization, such as concrete devices, concrete topics etc). This model can scale according to the needs, such ass adding more consumers, more event processing, additional concrete devices etc.{}

- Class Devices: a concrete implementation of the class device class be TCM 100, ACM 200, ACM 250. this class can be extended (scaled) to support additional devices, that may be of interest in the long tern.
- Class Gateway: the gateway is concretized with the RailXplore Core Shield Connect component. It can be replaced by other new gateways is required
- Class Published Topics: this class is concretized by the already well-defined topics such as Alarms, Events, Devices (including the information lists)
- Class Event Consumer: This class represents the subscribers or consumers of the subscription such as the customer systems (see more details below)

#### Architectural Design Pattern

The model below is based on the Software Design Pattern Observer/Observable, and the underlined implementation known as Producer/Consumer, Publisher/Subscriber (short form: Pub/Sub).

The following describes main workflow, which may vary depending on the requirements of the publisher, the subscriber, the environment, and the end-points ( from where the event is generated, such as in the hardware layer, till its consumed by the subscriber (customer system)

- A publisher application creates a topic in the Cloud Pub/Sub or On-premises service and sends messages to the topic.
- A message contains the required data and the underlined attributes that describe the event (event, alarm., device list etc)
- Messages can be persisted in a message database until they are delivered or may be deleted after a well-define time-out (such as in Apache Kafka, a possible implementation of the Sub/Pub pattern within a real-time data stream environments).
- Messages may be acknowledged by the subscribers.
- The Pub/Sub service forwards messages from a topic to all of its subscriptions.
- Each subscription receives messages either by the Pub/Sub pushing them to the subscriber's define endpoint (e.g. customer system), or by the subscriber pulling them from the service.
- The subscriber receives pending messages from its subscription and may acknowledges each one to the Pub/Sub service.

### API Reference

#### Data Types

| Device Type | Description                                                                                                                                                                         |
| ----------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Alarms      | APM detects various types of Alarms for track side assets. Each of it is published into ESB as a separate JSON message as soon as it is detected in APM                             |
| Events      | APM application can identify certain events like port status change, board state change etc. These events are published as a separate JSON message as soon as it is detected in APM |
| Alarms List | List of alarms detected over a period is published as a JSON Array                                                                                                                  |
| Event List  | List of events detected over a period is published as a JSON Array                                                                                                                  |
| Device List | List of active devices in APM                                                                                                                                                       |

#### Device Types

A customer (tenant) can subscribe for alarms, events and other device updates through the ESB for following device types:

- TCM100
- ACM200
- ACM250

#### Subscriptions

Based on the Data types and Device types, an administrator can create different subscriptions for a customer using the APM UI. Subscriptions can be Alarms, Events, Device List etc. An administrator can set up a subscription for a tenant during the installation time or it can be updated later using the ESB configuration screen.

The following are the available subscription types. Please refer user manual to get the detailed steps to enable and disable Subscriptions using APM UI.

| **Subscription Name**       | **Device Type** | **Subscription Type** | **Description**                                                                                                                                                                                     |
| --------------------------- | --------------- | --------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| TCM100 Live Alarms          | TCM100          | Alarms                | By enabling this subscription, APM application publishes TCM100 alarms to respective Kafka topic                                                                                                    |
| TCM100 Live Events          | TCM100          | Events                | By enabling this subscription, APM application publishes TCM100 events to respective Kafka topic as soon as it is identified in APM.                                                                |
| ACM200 Live Alarms          | ACM200          | Alarms                | By enabling this subscription, APM application publishes ACM200 alarms to respective Kafka topic as soon as it is identified in APM.                                                                |
| ACM200 Live Events          | ACM200          | Events                | By enabling this subscription, APM application publishes ACM200 events to respective Kafka topic as soon as it is identified in APM.                                                                |
| ACM250 Live Alarms          | ACM250          | Alarms                | By enabling this subscription, APM application publishes ACM250 alarms to respective Kafka topic as soon as it is identified in APM.                                                                |
| ACM250 Live Events          | ACM250          | Events                | By enabling this subscription, APM application publishes ACM250 events to respective Kafka topic as soon as it is identified in APM.                                                                |
| Periodic Active Alarms List | All             | Alarms List           | By enabling this subscription, APM application publishes all active alarms to a Kafka topic based on a schedule. Schedule is configurable, for example it can be run every 2 hours.                 |
| Periodic Device List        | All             | Device List           | By enabling this subscription, APM application publishes all active devices to a Kafka topic based on a schedule. Schedule is configurable, for example it can be run every 2 hours.                |
| Periodic Events List        | All             | Events List           | By enabling this subscription, APM application publishes all events identified in APM to a Kafka topic during a time window. Time window is configurable, for example it can be run in last 2 hour. |

#### ESB Connectivity Configuration

The ESB is currently implemented using Apache Kafka with real-time data streaming capabilities.

Customers can pull published Alarms, Events, and messages from the ESB by establishing connectivity to Kafka.

Connectivity to the ESB service can be established from the customer applications with the preferred technology and programming language. However, we recommend to use **NodeJS** and the related dependencies to build a consumer application as shown below. In the following we describe few aspects of the required packages, configuration, and code snippets to illustrate the concept and how it works. In addition, a full client running example in Node.js will be delivered to the customers, which can be reused and extended to fulfill any specific requirements.

##### Packages to install

install the following packages:

- Node.js and NPM (its package manager, available on its official website), for your specific operating system (WIndows, Linux, Mac, etc)
- Kafka client library for Node.js, such as kafkajs. NPM can be used for the installation as well, in a shell or in a terminal window run the following command:

```javascript
npm install kafkajs

```

##### Customer Application Configuration

Consumer application can be configured with the following details to establish connectivity to the ESB Kafka.

The consumer must be configured with at least one broker, in this case the ESB broker.

```javascript
// set the credentials, the secret, and the port during the installation, this information is used by kafla to trace back  the consumer
const ESBclientId = getClientId (setClientId());
const ESBclientSecret = getClientSecret (setClientSecret());
const ESBBrokerPort = getESBBrokerPort (setESBBrokerPor());

// Create broker list
const { ESBKafka } = require("kafkajs");

const ESBKafka = new Kafka({
  clientId: ESBclientSecret,
  brokers: [ESBkafka+":"+ESBBrokerPort],
});

```

#### SASL Mechanism

The sasl option can be used to configure the authentication mechanism.

```javascript
const SASL_Mechanism = 'plain';

```

### Kafka Schema Registry

Schema Registry offers a centralized location for the management and validation of schemas related to topic message data, facilitating the serialization and deserialization of the data transiting across the network. This ensure data consistency and compatibility compatibility.\
Using Avro schema in Kafka ensure speed and consistency, and efficiency.

Consumers can retrieve the topic schema from the registry.

```javascript
String ESBschemaRegistryUrl = getKafkaSchemaRegistryURL();
const ESBregistry = new SchemaRegistry(schemaRegistryUrl);

```

#### Enabling Authentication

this what's required to be done:

- Broker certificate is to be installed on each broker.
- Certificate is distributed to clients and installed in their truststore.
- Client certificate is installed on each client.
- Certificate is installed in each broker’s truststore.

#### ESBKafka Consumer Configuration

```javascript
// the first part is similar to the initial configuratiom, done once, here is to show how the instrauctions are related to eah other, including creating the consumer application
const { ESBKafka } = require("kafkajs");

// for testing on the local machine, having installed kafka and all related dependencies.  As a broker: localhost, is the equivalent of 127.0.0.1, port:9092, is standraf port for Kfaka, however, a differetn port can be selected 
const ESBKafka = new Kafka({
  clientId: "myCustomerID",
  brokers: ["localhost:9092"],
});

const ESBClientConsumer = kafka.consumer({ groupId: "customerGroup" });

```

#### Connect and Consume Messages

The following is a general example how to connect and consume messages

```javascript
conts ESBalarmTopic = "Alarms"; 
const run = async () => {
  await ESBclientConsumer.connect();
  await ESBclientConsumer.subscribe({ topic: ESBalarmTopic, fromBeginning: true });

  await ESBclientConsumer.run({
    eachMessage: async ({ ESBalarmTopic, partition, message }) => {
      console.log({
        value: message.value.toString(),
      });
    },
  });
};

```
# Senseye documentation

# Contact

If you have any enquiries or require support then please contact us directly:

Email: [support](mailto:senseye.support.industry@siemens.com)

- [Overview](overview.html)
- [Introduction](introduction/index.html)
  - [Authentication](introduction/authentication.html)
  - [External IDs](introduction/externalids.html)
  - [GraphQL](introduction/graphql.html)
  - [Security](introduction/security.html)
  - [Terminology](introduction/terminology.html)
  - [Time and Dates](introduction/timeanddates.html)
- [Machine](machine/index.html)
  - [Sensor](machine/sensor.html)
  - [Formats](machine/formats/_index.html)
    - [Timeseries](machine/formats/timeseries/_index.html)
      - [CSV](machine/formats/timeseries/csv.html)
      - [JSON](machine/formats/timeseries/json.html)
      - [Parquet](machine/formats/timeseries/parquet.html)
    - [Vibration](machine/formats/vibration/_index.html)
      - [Spectra](machine/formats/vibration/spectra/_index.html)
        - [CSV](machine/formats/vibration/spectra/csv.html)
        - [JSON](machine/formats/vibration/spectra/json.html)
        - [SKF](machine/formats/vibration/spectra/skf.html)
      - [Waveforms](machine/formats/vibration/waveforms/_index.html)
        - [CSV](machine/formats/vibration/waveforms/csv.html)
        - [JSON](machine/formats/vibration/waveforms/json.html)
  - [Pull](machine/pull/_index.html)
    - [Insights Hub](machine/pull/insightshub.html)
  - [Push](machine/push/_index.html)
    - [Blob](machine/push/blob.html)
    - [Email](machine/push/email.html)
    - [MQTT](machine/push/mqtt.html)
    - [WebDAV](machine/push/webdav.html)
    - [API](machine/push/api/_index.html)
      - [Timeseries](machine/push/api/timeseries.html)
      - [Vibration](machine/push/api/vibration.html)
- [Notifications](notifications/index.html)
  - [Pull](notifications/pull.html)
  - [Push](notifications/push.html)
- [Single Sign-On (SSO)](sso/index.html)
  - [ADFS](sso/ADFS.html)
- [Work Orders](workorders/index.html)
  - [Push](workorders/push.html)
  - [Structure](workorders/structure.html)
  - [Pull](workorders/pull/_index.html)
    - [Emaint](workorders/pull/emaint.html)
- [Contact](contact.html)

Welcome to Senseye's developer documentation! This site will describe how to integrate with Senseye, covering how to:

- provide [machine data](machine/index.html) and [work orders](workorders/index.html)
- be alerted to cases so that you can raise [work requests / notifications](notifications/index.html)
- configure [single sign on](sso/index.html)
- build custom integrations making use of our extensive [GraphQL API](introduction/graphql.html)

To keep up to date with changes, please sign up to our [developer mailing list](https://info.senseye.io/senseye-developer).

For further support, please contact [support](mailto:senseye.support.industry@siemens.com).

# Introduction

This section will introduce topics which are common across the various integration points.

If you plan to make use of Senseye's API, the following pages will be of interest:

- [Authentication](authentication.html) guide
- An introduction to our [GraphQL API](graphql.html), including information about its [public schema](https://graphqldocs.senseye.io)
- [External IDs](externalids.html) which allow the use of your IDs when interacting with Senseye's API

More generally, we provide information about the two [timestamp](timeanddates.html) formats used and the [security](security.html) of the Senseye system.

To access Senseye's APIs a valid access token needs to be provided by the client in the `Authorization` header of their request. There are currently two types of access tokens that can be used:

- Machine Token - A machine token is a temporary, or long-lived, token which is only allowed to perform specific actions and is not associated with a user
- User Token - A user token is a temporary token which is associated with a particular user, with all the permissions of that user

Both types of token can be used, however it is recommended to use machine tokens for integration purposes (machine-machine interfaces) and user tokens for performing actions as, or on behalf of, a user.

Please note, each token uses a different standard for encoding information. Machine tokens use [paseto v4](https://paseto.io/) while user tokens use [jwt](https://jwt.io/).

## Machine Tokens

Machine tokens can only be generated by Senseye support. If you require a machine token please log a support ticket in our [support centre](https://help.senseye.io/hc/en-gb/requests/new) or by emailing [support](mailto:senseye.support.industry@siemens.com), noting your organization, which actions the token should permit the bearer to perform, a name for the token, and an expiry date for the token if the token should not be permanent. A table of available permissions can be found below.

| Permission          | Purpose                                                                  |
| ------------------- | ------------------------------------------------------------------------ |
| Internal Hub Write  | Storing machine time series and frequency data in Senseye's internal hub |
| Internal Hub Create | Create sensors in Senseye's internal hub                                 |
| External ID Read    | Read external IDs for objects in Senseye                                 |
| External ID Write   | Set external IDs for objects in Senseye                                  |

You can request that a previously generated machine token be invalidated by logging a support ticket referencing the name of the token.

## User Tokens

Senseye provides an authentication `https://api.senseye.io/oauth/token` to request a user token, which uses your Senseye username and password to authenticate. A user token can be requested by making a POST request to the authentication endpoint with the following details in x-www-form-urlencoded format:

- grant_type: password
- username: Your Senseye Username
- password: Your Senseye Password

Please note, as the contents are expected to be URL encoded you may need to manually encode your credentials depending on the client you are using.

```http
POST /oauth/token HTTP/1.1
Host: api.senseye.io
Content-Type: application/x-www-form-urlencoded

grant_type=password&username=senseye_username&password=senseye_password

```

```bash
curl --location --request POST 'https://api.senseye.io/oauth/token' \
--header 'Content-Type: application/x-www-form-urlencoded' \
--data-urlencode 'grant_type=password' \
--data-urlencode 'username=senseye_username' \
--data-urlencode 'password=senseye_password'

```

This provides a JSON response containing the [JSON Web Token (JWT)](https://jwt.io/) based access token:

```json
{
    "access_token": "your_access_token",
    "expires_in": 86400,
    "token_type": "Bearer"
}

```

The `expires_in` key of the returned json specifies how long the provided user token is valid for in seconds.

It is strongly recommended that a user token is cached and the same token is used for requests sent during the lifetime of the token. Once a token has expired, a new user token can be requested and similarly cached. To avoid failed requests, it is recommended to request a new token (known as refreshing the token) a minute before it expires.

Please note, Senseye's authentication endpoint `https://api.senseye.io/oauth/token` is rate limited. If a user exceeds the rate limit a HTTP 429 ([Too Many Requests](http://tools.ietf.org/html/rfc6585#section-4)) status code will be returned.

Alternatively if you are trying to request a token using a 3rd party HTTP request client (such as Postman or Insomnia), the endpoint conforms to the Resource Owner Password Credentials [OAuth 2.0](https://oauth.net/2/) standard. This is supported directly by most HTTP request clients. You will need to provide the following parameters when authenticating:

- Token URL: `https://api.senseye.io/oauth/token`
- Username: Your Senseye username
- Password: Your Senseye password

## Making Requests

Requests to Senseye's APIs should include an access token in the `Authorization` header prefixed with `Bearer`. As an example, the following requests your organization's name from our [GraphQL API](graphql.html):

```http
POST /v1/graphql HTTP/1.1
Host: api.senseye.io
Authorization: Bearer your_access_token
Content-Type: application/json

{"query":"{organization {name}}","variables":{}}

```

```bash
curl --location --request POST 'https://api.senseye.io/v1/graphql' \
--header 'Authorization: Bearer your_access_token' \
--header 'Content-Type: application/json' \
--data-raw '{"query":"{organization{name}}","variables":{}}'

```

When integrating external systems with Senseye, there is often a requirement to map from Senseye's IDs to an ID within the external system. Senseye supports this mapping with External IDs; this allows one or more IDs from external systems to be mapped to Senseye IDs. External IDs can be configured on any node within the hierarchy, including Assets, Sensors and Sub-Levels.

The following describes how to make use of External IDs within the [GraphQL API](graphql.html) when using a single external system. If you have multiple external systems which you want to map to a given node within the Senseye hierarchy, please contact [support](mailto:senseye.support.industry@siemens.com) to setup the required configuration and provide further API documentation if required for your use case.

## Setting an External ID

The [setExternalID mutation](https://graphqldocs.senseye.io/#mutation-setExternalId) can be used to set an External ID for a given Senseye ID:

Query

```graphql
mutation SetExternalID($input: SetExternalIdInput!) {
    setExternalId(input: $input) {
        externalId
        id
    }
}

```

Variables

```json
{
    "input": {
        "externalId": "External System's ID",
        "id": "Senseye's ID"
    }
}

```

As discussed above, the Senseye ID could be any hierarchy node - the node type used will depend on the type of integration and how closely the hierarchies within both systems match.

To set an External ID, you must have write permissions on the node.

## Using an External ID

Once set, hierarchy nodes can be retrieved using the External ID. The example below uses the [hierarchyNodeByExternalID query](https://graphqldocs.senseye.io/#query-hierarchyNodeByExternalID) to gather the Senseye ID and name of the hierarchy node with a given External ID. Furthermore, if the node is an Asset, it will gather its current attention index.

Query

```graphql
query {
    hierarchyNodeByExternalID(externalID: "External System's ID") {
        id
        name
        ... on Asset {
            currentAttentionIndex {
                value
                timestamp
            }
        }
    }
}

```

If you are setting External IDs just on Assets, there is an Asset specific query ([assetByExternalID](https://graphqldocs.senseye.io/#query-assetByExternalID)) which avoids the type casting seen above:

Query

```graphql
query {
    assetByExternalID(externalID: "External System's ID") {
        id
        name
        currentAttentionIndex {
            value
            timestamp
        }
    }
}

```

All [HierarchyNode](https://graphqldocs.senseye.io/#definition-HierarchyNode) types allow their External ID to be retrieved using the `externalId` attribute. For example, the following query will gather 10 Cases from within your organization, and for each, return the date when the Case was opened and the associated Asset's External ID:

Query

```graphql
query {
    organization {
        cases(limit: 10) {
            results {
                openedAt
                asset {
                    externalId
                }
            }
        }
    }
}

```

Senseye provides a GraphQL API which covers the majority of the functionality available within Senseye PdM.

GraphQL is a query language designed for APIs, it allows clients to gather many resources in a single request whilst providing just the information that is requested. The API conforms to a [strongly typed schema](https://graphqldocs.senseye.io/), which acts as a contract between the client and the server.

More information about GraphQL is available on their [website](https://graphql.org/learn/).

## Schema

Senseye's GraphQL schema can be found at .

Along with the root level [queries](https://graphqldocs.senseye.io/#group-Operations-Queries) and [mutations](https://graphqldocs.senseye.io/#group-Operations-Mutations), many common operations belong to hierarchy objects - the [Organisation](https://graphqldocs.senseye.io/#definition-Organization) and [Asset](https://graphqldocs.senseye.io/#definition-Asset) objects are good examples.

## Authentication

Each request to the GraphQL API should include an access token within its `Authorization` header. For more information, please visit the [Authentication](authentication.html) documentation.

## Transport

Senseye exposes its GraphQL endpoint via HTTP at the following URL: `https://api.senseye.io/v1/graphql`. This accepts POST requests with GraphQL queries encoded into the request's body as JSON. HTTP is a common way to expose GraphQL APIs and is described in more detail within the [GraphQL documentation](https://graphql.org/learn/serving-over-http).

As an example, the following GraphQL query gathers your organization's name:

```graphql
{
    organization {
        name
    }
}

```

this query can be executed via HTTP using the following request:

```http
POST /v1/graphql HTTP/1.1
Host: api.senseye.io
Authorization: Bearer your_access_token
Content-Type: application/json

{"query":"{organization {name}}","variables":{}}

```

```bash
curl --location --request POST 'https://api.senseye.io/v1/graphql' \
--header 'Authorization: Bearer your_access_token' \
--header 'Content-Type: application/json' \
--data-raw '{"query":"{organization{name}}","variables":{}}'

```

The response will be of the following form:

```json
{
    "data": {
        "organization": {
            "name": "your organization's name"
        }
    }
}

```

### Errors

Unlike REST APIs, GraphQL APIs do not make use of status codes - **all requests will return with a 200 status**. If a request fails, the response will include an `errors` array, which will describe the errors which occurred. For example:

Response

```json
{
    "errors": [
        {
            "message": "The request failed because the failure mode was of an incorrect type.",
            "path": [
                "createWorkEvent"
            ],
            "extensions": {
                "code": "WORK_EVENT_UNEXPECTED_FAILURE_MODE",
                "errorid": "cf6c6547-4e7e-4a74-997a-5b625e99e1a7",
                "failureMode": "BAD_FAILURE_MODE"
            }
        }
    ],
    "data": {
        "createWorkEvent": null
    }
}

```

In the `extensions` object will be an `errorid` field. Should you need help with resolving the error, please contact [support](mailto:senseye.support.industry@siemens.com) and quote the `errorid` reference.

The `extensions` object also contains a `code`. These codes are described in more detail [here](https://graphqldocs.senseye.io/#definition-SenseyeError)

## Pagination

Some functions support pagination to ensure queries do not become too large. For example, `cases(offset:2, limit:5)` returns 5 case files starting at the third in the list. This function will also return a `paginationDetails` structure in its result, which includes the following information:

- `total`: the total number of results that are available
- `count`: the actual number of results that were returned in this set
- `offset`: the offset into the total result set
- `limit`: the actual limit that was used in creating the result. The API Gateway may restrict this, so it may be less than the limit requested.

To request the next page of results, you would set the `offset` to be `offset + count`, and the `limit` to be `limit`. If `offset + count` is greater than `total` then the current page is the final page.

Senseye goes to great lengths to ensure that working with us is safe, reliable and sustainable. From initial deployment to enterprise-wide scale, your machine and maintenance data is in safe hands thanks to our robust internal systems, independent testing & verification, standards compliance, and strict staff policies.

To find out more, please view our [security white paper](https://siemens.com/senseye-security).

Senseye is a predictive maintenance system designed to monitor industrial machinery to raise insights on the condition, and any potential degradation, of these machines. This page is a collection of common terms used within Senseye.

## Asset

An object being monitored by Senseye, in most cases this is an individual piece of industrial machinery such as a motor or gearbox. However, if limited monitoring data is available this can be a collection of machines, such as a drive chain, or if an abundance of monitoring data is available can be components of an industrial machine, such as motor windings.

Assets form the notification level of Senseye, these notifications are known as [cases](#case), and are raised when an asset's [Attention Index](#attention-index) is raised to the medium or high level. Only one case can be open against the asset at a time. Cases contain evidence in the form of [insights](#insight) generated from the [measures](#measure) associated with the [sensors](#sensor) attached to the asset.

Assets are organized in the [hierarchy](#hierarchy), where they can be children of [sublevels](#sublevel). An asset cannot be the child of another asset.

Assets are identified by a unique [UUID](#uuid).

## Attention Index

A score for the amount of attention needed on an [asset](#asset). This score is generated using a machine learning algorithm based on [insights](#insight) raised by Senseye. When the Attention Index reaches Medium or High a case is raised on the [asset](#asset).

The Attention Index is always between 0 and 1 inclusive, and categorized below:

- Normal: 0 - 0.25
- Low: 0.25 - 0.5
- Medium: 0.5 - 0.75
- High: 0.75 - 1

## Case

A notification within Senseye. Cases are opened when the [Attention Index](#attention-index) on an [asset](#asset) goes into Medium or High. Each case has associated case evidence, containing the [insights](#insight) that contributed to the case being opened. Only one case can be open on an [asset](#asset) at a time.

Once a case has been handled it can be closed. Closing a case requires feedback to be provided using the latest feedback questions.

Cases can be associated to a [user](#user) via case acknowledgement.

Cases are identified by a unique [UUID](#uuid).

## Event

Additional timestamped information against an [asset](#asset). Events are used to record timestamped information such as investigation notes, logs of maintenance work, or recording of KPIs. There are multiple types of event, the most commonly used being:

- Work Event: Record of Maintenance Work Done
- Note: Additional free type information
- Manual Measurement: Log of an manual measurement
- KPI: Record of resources saved/used

Events are identified by a unique [UUID](#uuid).

## External ID

A unique mapping between an object in Senseye and a unique reference. Most commonly used to map an [UUID](#uuid), such as an [asset ID](#asset), to a client's ID for the object, such as the machine's location ID.

Each external ID has three parts:

- Type: what system the mapping relates to
  - NODE: used for [hierarchy](#hierarchy) nodes
  - SENSOR: used for [sensors](#sensor) when sending [machine data](../machine/index.html)
- Internal ID: a unique internal reference, commonly a [UUID](#uuid)
- External ID: a unique external reference string

## Hierarchy

Structure of an [organization](#organization). Each hierarchy is made up of nodes, of which there are four types:

- [Organization](#organization)
  - Representative of the user's tenant
  - Always at the top level of the hierarchy
- [Sublevel](#sublevel)
  - A level of the hierarchy
  - Used to add structure/grouping to the hierarchy, similar to a folder
- [Asset](#asset)
  - The notification level of the hierarchy
  - Can be a child of a sublevel or an organization
- [Sensor](#sensor)
  - The data level of the hierarchy
  - Can only be a child of an asset

All nodes can be identified by a unique [UUID](#uuid).

For [sublevels](#sublevel), [assets](#asset), and [sensors](#sensor) [external IDs](#external-id) of type NODE can be setup. This can be used to query the Senseye hierarchy based on client reference IDs rather than [UUIDs](#uuid).

## Insight

Detection of a change on a [measure](#measure), also called patterns. Insights are the primary output of Senseye's algorithms and are detections of changes on [measures](#measure). There are several different types of insights, corresponding to different forms of detection:

- Anomalies
- Trends
- Threshold Violations
- Forecasted Threshold Violations
- Degradations
- Failure Matches
- Missing Data (Data Issue)
- Flatlining (Data Issue)

Missing Data and Flatlining insights cannot raise a [case](#case) but are used for detection of data issues, rather than machine degradation.

Insights are identified by a unique [UUID](#uuid).

## Measure

An individually named time series. Measures form the basis upon which [insights](#insight) are detected. They are automatically generated from the time series data sent to a [sensor](#sensor), appearing shortly after first being sent to the sensor.

Measures are identified with their fully qualified name (FQN), which takes the form of `measure:{sensor-id}:{measure-source-name}`. The measure source name is the name of the measure as sent in the time series data or is proscribed in the case of measures extracted from frequency data. See [machine data](../machine/index.html) for more information.

## Organization

A customers tenant. Senseye is multi-tenanted, with each of the individual tenants being referred to as an organization. Each organization is a self contained space, with it's own [hierarchy](#hierarchy), [cases](#case), and [users](#user).

An organization is identified with a group UUID, where `group|` prefixes the organization's UUID. In all cases an organization is associated to the user's authentication and the organization will not need to be specified.

## Sensor

A logical grouping of [measures](#measure). Sensors were originally meant to reflect physical sensors on [assets](#asset) which may report multiple [measures](#measure). Today sensors can be any logical grouping of [measures](#measure) that need to be associated with an [asset](#asset).

Sensors have an 'ingest' property which determines what type of data they can accept. A 'timeseries' sensor can ingest raw time series data, for example scalar data measured directly from temperature, oil, pressure sensors. Alternatively, a 'frequency' sensor will only ingest frequency spectra files, which will undergo postprocessing to extract [measures](#measure).

Sensors are attached to [assets](#asset) to join the [hierarchy](#hierarchy), once attached the data from the [measures](#measure) on the sensor are used for monitoring the [asset](#asset).

Sensors that are not attached are known as unattached sensors. They can be found by querying for the unattached sensors in the [organization](#organization).

Each sensor can only be attached to one [asset](#asset).

[Measures](#measure) are automatically generated based on the [machine data](../machine/index.html) sent to the sensor.

Sensors are identified with a unique [UUID](#uuid). For the purposes of sending [machine data](../machine/index.html) an [external ID](#external-id) of type SENSOR, otherwise known as a mapping ID, can be setup.

## Sublevel

A level of the hierarchy under which further sublevels or [assets](#asset) can be contained. Sublevels are often used to organize [assets](#asset) into logical groupings within Senseye, and often represent physical locations, such as a site or area.

## User

An individual's account in Senseye. Users action [cases](#case) and feedback to Senseye. Where a user takes an action relating to a [case](#case) or leaves an [event](#event) on an [asset](#asset) they will be recorded as the author.

Users are identified by a unique ID, constructed from identity provider specific details joined by `|`.

## UUID

A 128-bit identifier as specified in [RFC 4122](https://www.ietf.org/rfc/rfc4122.txt).

UUIDs are specified in the 8-4-4-4-12 format, like xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx.

## RFC3339

In the vast majority of APIs, Senseye uses the RFC3339 standard to represent dates and times.

An example of date and time conforming to RFC3339 is: `2022-12-05T13:22:38.000-08:00`

For completeness, the RFC3339 specification is [here](https://www.rfc-editor.org/rfc/rfc3339)

Formally, the syntax is

```text
   date-fullyear   = 4DIGIT
   date-month      = 2DIGIT  ; 01-12
   date-mday       = 2DIGIT  ; 01-28, 01-29, 01-30, 01-31 based on
                             ; month/year
   time-hour       = 2DIGIT  ; 00-23
   time-minute     = 2DIGIT  ; 00-59
   time-second     = 2DIGIT  ; 00-58, 00-59, 00-60 based on leap second
                             ; rules
   time-secfrac    = "." 1*DIGIT
   time-numoffset  = ("+" / "-") time-hour ":" time-minute
   time-offset     = "Z" / time-numoffset

   partial-time    = time-hour ":" time-minute ":" time-second
                     [time-secfrac]
   full-date       = date-fullyear "-" date-month "-" date-mday
   full-time       = partial-time time-offset

   date-time       = full-date "T" full-time

```

There are two aspects to draw attention to:

- the Date format is `yyyy-mm-dd` i.e. the year, the month and then the day
- there is a time offset at the end. In the above example, `-08:00` is used which represents UTC-8 (Pacific Standard Time). For UTC, `+00:00` can be used or `Z` which stands for the Zero or Zulu timezone (UTC+0).

Senseye stores data in UTC+0 time.

## Unix Time

A commonly used lightweight protocol to convey data about a sensor is `SenML`, which is described [here](https://datatracker.ietf.org/doc/html/rfc8428). This standard allows time to be represented as the number of seconds after January 1st, 1970 at UTC.

Unix Time is not used in Senseye's GraphQL API, but is used within the [JSON time series data API](../machine/push/api/timeseries.html).

# Machine Data

This section describes how to integrate machine data with Senseye. Machine data is a broad term that covers all data to be used for predictive maintenance purposes, whether a condition indicator or contextual data.

Senseye includes a cloud data storage solution which can be integrated with via several methods:

- [API](push/api/_index.html)
- [MQTT](push/mqtt.html)
- [E-mail](push/email.html)
- [S3 / Azure Blob Storage](push/blob.html)

Alternatively, Senseye can connect to your historian to collect the data directly.

Currently, we support two types of machine data:

- Time Series Data
- Vibration Data

## Time Series Data

Senseye supports the ingestion of time series data with a maximum frequency for a single time series (known as a [measure](../introduction/terminology.html#measure)) of 1Hz, which can be either a condition indicator or contextual data. Condition indicators, along with relevant contextual data, are used by Senseye's analytics to monitor machines (known as [assets](../introduction/terminology.html#asset)) for signs of degradation. For data sampled more frequently than 1Hz preprocessing is required on the data to downsample it so it can be ingested into Senseye.

## Vibration Data

Senseye supports the ingestion of velocity and acceleration waveforms, and can process these into frequency spectra. Alternatively, if the raw waveform is not available, we can ingest spectra directly. Our system can handle either CSV or JSON in our own formats, or we additionally have built-in support for exports from third-party software.

From raw waveforms, we can extract common condition indicators such as the RMS, Peak-to-Peak and Zero-to-Peak values. When Senseye receives a frequency spectrum or processes a waveform into one, we attempt to extract the amplitude at key frequencies as measures, which are then trackable over time as additional condition indicators of your machinery. Currently, we will extract the amplitude matching the rotational speed of your machine and it's harmonics, which we label as "1x" - "5x". The rotational speed can be sent along with the sample, or alternatively we can allow a fixed speed on a per-sensor basis so that we can perform this calculation. We do not currently support extracting amplitudes at additional key frequencies.

Our method for extracting the amplitude at a given frequency is to take a window of 1% of the target frequency and take the maximum value within this range. So for example, if we had measured amplitudes of 1.1 and 1.2 at 100Hz and 101Hz, and our target frequency was 100.5Hz, we would extract the value of 1.2.

Creating an internal hub sensor can be done via either our [GraphQL](../introduction/graphql.html) or JSON REST APIs. Sensors can either ingest time series data or frequency spectrum data; they cannot do both, and this must be set when a sensor is being created.

During sensor creation a `mappingId` can be provided, which allows you to interact with the [sensor](../introduction/terminology.html#sensor) using your own ID, rather than Senseye's ID. If you plan to use a `mappingID` to send data we recommend skipping the sensor creation step as sending data to a new `mappingId` will automatically create a new sensor for you. For more information, see the [Push to Senseye](push/_index.html) page.

You may also optionally provide a description for the sensor.

## GraphQL

We support sensor creation via [GraphQL](../introduction/graphql.html) using the [createSensor mutation](https://graphqldocs.senseye.io/#mutation-createSensor).

Query

```graphql
mutation createSensor($input: CreateSensorInput!) {
    createSensor(input: $input) {
        sensor {
            id
            name
            mappingId
        }
    }
}

```

Variables

```json
{
    "input": {
        "hubType": "IHUB",
        "name": "Sensor's name",
        "description": "A description of the sensor",
        "mappingId": "Your ID for the sensor"
    }
}

```

Response

```json
{
    "data": {
        "createSensor": {
            "sensor": {
                "id": "Senseye's ID for the sensor",
                "name": "Sensor's name",
                "mappingId": "Your ID for the sensor",
                "ingestType": "timeseries" | "frequency"
            }
        }
    }
}

```

## JSON REST

Alternatively, sensors can be created via our REST endpoint. This can be done by sending a POST request with a JSON body containing the sensor's details, as shown below:

```http
POST /v1/hub/sensors HTTP/1.1
Host: api.senseye.io
Authorization: Bearer your_access_token
Content-Type: application/json

{
    "name": "Sensor's name",
    "description": "A description of the sensor", // optional
    "mappingId": "Your ID for the sensor", // optional
    "ingest": "timeseries" | "frequency"
}

```

```bash
curl --location --request POST 'https://api.senseye.io/v1/hub/sensors' \
--header 'Authorization: Bearer your_access_token' \
--header 'Content-Type: application/json' \
--data-raw '{
    "name": "Sensor name",
    "description": "A description of the sensor",
    "mappingId": "Your ID for the sensor",
    "ingest": "timeseries" | "frequency"
}'

```

# Data Formats

This section describes the format of file based transfers of time series and vibration data.

# Time Series Data Formats

This section describes the format of file based transfers of time series data. We support [CSV](csv.html), [Parquet](parquet.html) and [JSON](json.html).

## Structure

Senseye adopts a wide time series format, with a row per timestamp, and a column per measure:

example@1651147200.csv

```text
DATETIME,temperature,current,product,run
2022-04-28T12:00:00+02:00,20,516,"A24",true
2022-04-28T12:30:15+02:00,21,489,"A24",true
2022-04-28T12:45:55+02:00,,502,"A24",false
2022-04-28T13:14:00+02:00,21,503,"A23",true

```

A file version of this example is available [here](files/example%401651147200.csv).

The following describes the structure in further detail:

- The first row must be a header, the first cell being `DATETIME` and the subsequent cells being the measures being recorded.
- Subsequent rows collect data across the measures for a given timestamp.
- Not every measure requires a data point per timestamp, in which case the cell should remain empty.
- The DATETIME column should contain RFC3339 dates and times, including timezone - more details on the required format are available [here](../../../introduction/timeanddates.html).
- Measure names must not be duplicated within the file (i.e. two columns shouldn't have the same header value).
- Numbers must use a period (`.`) as the decimal separator.
- Strings should be provided in quotation marks (`"`). Numeric strings, or strings containing scientific representations of numbers (`"3e+00"`), will be converted to numbers. However, we strongly suggest any number be sent as a number rather than in a string.
- Boolean values should be either `true` or `false`
- Each row must have the same number of columns as the header row.

## File

### Name

File names should be made up of two parts: the mapping ID for the sensor the data is for and a unique value to avoid collisions, separated by `@`. As an example: `motorM32@1651149458.csv`.

The initial section of the filename is used to route the data to the correct sensor based on the mapping ID. If a new mapping ID/filename is provided, Senseye will provision a new sensor associated with this mapping ID and route all future files using this mapping ID as the filename to the sensor. In the above example, the mapping ID would be `motorM32`.

The later section is included to avoid file collisions. We recommend using a unix timestamp, but any unique value is suitable. In the above example, the unix timestamp is `1651149458`.

### Size

Please limit CSV files to no more than 1MB each.

### Encoding

The data file must use UTF-8 character encoding.

## Structure

Senseye's JSON format allows data from multiple assets to be transferred within a single file.

example@1651147200.json

```json
[
    {
        "assetId": "mappingId",
        "dataPoints": [
            {
                "timestamp": "2022-04-28T12:00:00+02:00",
                "measures": [
                    {
                        "name": "temperature",
                        "value": 20
                    },
                    {
                        "name": "current",
                        "value": 516
                    },
                    {
                        "name": "product",
                        "value": "A24"
                    },
                    {
                        "name": "run",
                        "value": true
                    }
                ]
            },
            {
                "timestamp": "2022-04-28T12:45:55+02:00",
                "measures": [
                    {
                        "name": "current",
                        "value": 502
                    }
                ]
            }
        ]
    }
]

```

example@1651147200.json

```json
[{
    "sensorId": "sensorId",
    "dataPoints": [{
        "timestamp": "2022-04-28T12:00:00+02:00",
        "measures": [{
            "name": "temperature",
            "value": 20
        },{
            "name": "current",
            "value": 516
        },{
            "name": "product",
            "value": "A24"
        },{
            "name": "run",
            "value": true
        }]
    }, {
        "timestamp": "2022-04-28T12:45:55+02:00",
        "measures": [{
            "name": "current",
            "value": 502
        }]
    }]
}]

```

A larger example with multiple assets can be downloaded [here](files/example%401651147200.json).

The following describes the structure in further detail, in addition, a [JSON Schema definition](files/json-file-schema.json) is available.

Root:

- The root of the JSON should be an array which contains data for one or more sensors - see 'Sensor Data'

Sensor Data:

- `assetId`: a unique mapping ID for a sensor. If a sensor with the mapping ID does not exist, a new sensor will be provisioned.
- `sensorId`: a UUID that Senseye has allocated to a sensor. This field is used only if the sensor already exists.
- `datapoints`: an array of datapoints obtained from the asset - see 'Datapoint'. There should be one entry per timestamp.

**NOTE**: Please provide either assetId or sensorId, but not both.

Datapoint:

- `timestamp`: should be a string conforming with RFC3339 dates and times, including timezone - more details on the required format are available [here](../../../introduction/timeanddates.html).
- `measures`: an array measurement values for the given timestamp - see 'Measurement',

Measurement:

- `name`: a string name given to the measurement. This value will be the name associated with the time series within Senseye.
- `value`: the value of the measure. Can be either a number, boolean or string. Floating point numbers should be in the format 1.234e-56.

## File

### Name

To avoid file collisions, the name of the file should be unique. To aid debugging, we recommend composing the file name using two components: an ID for the asset or group of assets being monitored, and a unix timestamp, separated by `@` - as an example: `motors@1651149458.json`.

### Size

Please limit JSON files to no more than 1MB.

### Encoding

The data file must use UTF-8 character encoding.

[Apache Parquet](https://parquet.apache.org/) is a free and open-source column-oriented data storage format.

## Structure

The expected schema for parquet files is as follows:

schema.parquet

```yaml
assetId: { type: 'UTF8' }
measures: {
    repeated: true,
    fields: {
        name: { type: 'UTF8' }
    }
}
data: {
    repeated: true,
    fields: {
        time: { type: 'UTF8' },
        measureName: { type: 'UTF8' },
        value: { type: 'DOUBLE', optional: true },
        stringValue: { type: 'UTF8', optional: true }
    }
}

```

The root of schema is an object which contains multiple fields:

- `assetId`: a UTF8 string containing a unique mapping ID for a sensor. If a sensor with the mapping ID does not exist, a new sensor will be automatically provisioned.
- `measures`: an array of the measure names included within the data array.
- `data`: an array of data points. There should be one entry per timestamp.
- `data.time`: the timestamp of the entry. Should be a UTF8 string conforming with RFC3339 dates and times, including timezone - more details on the required format are available [here](../../../introduction/timeanddates.html).
- `data.measureName`: a string name given to the measurement. This value will be the name associated with the time series within Senseye.
- `data.value` or `data.stringValue`: the value of the measure. This can be either a double-precision float number or a string.

## File

### Name

To avoid file collisions, the name of the file should be unique. To aid debugging, we recommend composing the file name using two components: an ID for the asset being monitored, and a unix timestamp, separated by `@` - as an example: `motorM23@1651149458.parquet`.

### Size

Please limit Parquet files to no more than 1MB.

### Version

The file must be exactly parquet format version 1.0.

### Encoding

The data file must use the parquet binary format and be encoded with one of the following supported codecs:

- PLAIN
- RLE

### Compression

The supported compression methods are:

- UNCOMPRESSED
- GZIP
- SNAPPY
- LZO
- BROTLI

# Vibration Data Formats

This section describes the format of file based transfers of vibration data.

We support ingesting velocity and acceleration waveforms, and can process these into frequency spectra. Alternatively, if the raw waveform is not available, we can ingest spectra directly. Our system can handle either CSV or JSON in our own formats, or we additionally have built-in support for exports from third-party software.

# Spectra Data Formats

This section describes the format of file based transfers of frequency spectrum data. We support [CSV](csv.html) and [JSON](json.html). We additionally support third party formats; currently this includes [SKF @ptitude Analyst CSV files](skf.html).

## Structure

Each CSV file can contain a number of frequency spectra in the following format:

```text
timestamp,amplitudeUnit,measurementType,spacing,end,number,speed,data
2024-03-04T18:13:00Z,mm/s,velocity,1,4,5,2,"0,1,2,3,4"

```

A file version of this example is available [here](../files/example%401651147200.csv).

The following describes the structure in further detail:

- The first row must be a header. The fields are all compulsory.
- The `timestamp` column should contain RFC3339 dates and times, including timezone - more details on the required format are available [here](../../../../introduction/timeanddates.html).
- Each row should contain a frequency spectrum. It is assumed that the data starts at 0 Hz.
- The `amplitudeUnit` should be a string value representing the units of measurement for the amplitude. For velocity, 'in/s', 'mm/s' and 'm/s' are valid units. For acceleration, 'in/s^2', 'mm/s^2' and 'm/s^2' are valid units. Conversion must be undertaken to one of these units prior to sending data to Senseye.
- The `measurementType` should be a string value of either `velocity` or `acceleration`.
- The `spacing` details the gap between bins in Hz i.e. here 1Hz
- The `end` details the frequency at the last value in Hz i.e. here 4Hz
- The `number` is the number of points in the spectrum i.e. here 5.
- The `speed` is the speed of the machine, or reference frequency, in Hz i.e. here 2Hz
- The `data` field is a comma separated list of points, wrapped in escaped quote marks. The data is interpreted as a list of 64-bit floating point values. The number of these must match the number parameter.
- Numbers must use a period (`.`) as the decimal separator.
- Each row must have the same number of columns as the header row.

## File

### Name

File names should be made up of two parts: the mapping ID for the sensor the data is for and a unique value to avoid collisions, separated by `@`. As an example: `motorM32@1651149458.csv`.

The initial section of the filename is used to route the data to the correct sensor based on the mapping ID. If a new mapping ID/filename is provided, Senseye will provision a new sensor associated with this mapping ID and route all future files using this mapping ID as the filename to the sensor. In the above example, the mapping ID would be `motorM32`.

The later section is included to avoid file collisions. We recommend using a unix timestamp, but any unique value is suitable. In the above example, the unix timestamp is `1651149458`.

### Size

Please limit CSV files to no more than 10MB each.

### Encoding

The data file must use UTF-8 character encoding.

## Structure

The JSON format for frequency spectra is as follows:

```json
[
    {
        "mappingId": "mappingId",
        "timestamp": "2024-03-04T18:13:00Z",
        "amplitudeUnit": "mm/s",
        "spacing": 1.0,
        "end": 4.0,
        "number": 5,
        "data": [
            0,
            1,
            2,
            3,
            4
        ],
        "speed": 1.0,
        "measurementType": "acceleration"
        |
        "velocity"
    }
]

```

```json
[{
    "sensorId":         "sensorId",
    "timestamp":        "2024-03-04T18:13:00Z",
    "amplitudeUnit":    "mm/s",
    "spacing":          1.0,
    "end":              4.0,
    "number":           5,
    "data":             [0, 1, 2, 3, 4],
    "speed":            1.0,
    "measurementType":  "acceleration" | "velocity"
}]

```

- The `mappingId` is a unique mapping ID for a sensor. If a sensor with the mapping ID does not exist, a new sensor will be provisioned.
- The `sensorId` is a UUID that Senseye has allocated to a sensor. This field is used only if the sensor already exists.
- The `timestamp` should be in RFC3339 dates and times, including timezone - more details on the required format are available [here](../../../../introduction/timeanddates.html).
- It is assumed that the data starts at 0 Hz.
- The `amplitudeUnit` should be a string value representing the units of measurement for the amplitude. For velocity, 'in/s', 'mm/s' and 'm/s' are valid units. For acceleration, 'in/s^2', 'mm/s^2' and 'm/s^2' are valid units. Conversion must be undertaken to one of these units prior to sending data to Senseye.
- The `spacing` details the gap between bins in Hz i.e. here 1Hz
- The `end` details the frequency at the last value in Hz i.e. here 4Hz
- The `number` is the number of points in the spectrum i.e. here 5.
- The `speed` is the speed of the machine, or reference frequency, in Hz i.e. here 2Hz
- The `data` field is a comma separated list of points, wrapped in escaped quote marks. The data is interpreted as a list of 64-bit floating point values. The number of these must match the number parameter.

## File

### Name

To avoid file collisions, the name of the file should be unique. To aid debugging, we recommend composing the file name using two components: an ID for the asset or group of assets being monitored, and a unix timestamp, separated by `@` - as an example: `motors@1651149458.json`.

### Size

Please limit the JSON request body to no more than 10MB.

We can ingest SKF @ptitude Analyst outputted frequency spectra in the following format:

```text
"Point Path","DTS","Unit","Detection","Channel","Lines","End Frequency","Speed (Hz)","Process Value","Unit","BOV / GAP","DIGITAL","Data"
"NodeID","2023-02-21 9:42:39 AM","mm/sec","Peak","1","5","4","2","N/A","N/A","N/A","N/A","1","2","3","4","5"

```

We require all fields here to be present and unmodified before ingestion - please note that opening and re-saving outputted files in Microsoft Excel may cause issues with ingestion.

# Waveform Data Formats

This section describes the format of file based transfers of vibration waveform data. We support [CSV](csv.html) and [JSON](json.html).

## Structure

Each CSV file can contain a number of waveforms in the following format:

```text
timestamp,amplitudeUnit,measurementType,spacing,end,number,speed,data
2024-03-04T18:13:00Z,mm/s,velocity,1,4,5,2,"0,1,2,3,4"

```

A file version of this example is available [here](../files/example%401651147200.csv).

The following describes the structure in further detail:

- The first row must be a header. The fields are all compulsory.
- The `timestamp` column should contain RFC3339 dates and times, including timezone - more details on the required format are available [here](../../../../introduction/timeanddates.html).
- Each row should contain a frequency spectrum. It is assumed that the data starts at 0 Hz.
- The `amplitudeUnit` should be a string value representing the units of measurement for the amplitude. For velocity, 'in/s', 'mm/s' and 'm/s' are valid units. For acceleration, 'in/s^2', 'mm/s^2' and 'm/s^2' are valid units. Conversion must be undertaken to one of these units prior to sending data to Senseye.
- The `measurementType` should be a string value of either 'velocity' or 'acceleration'.
- The `spacing` details the gap between bins in seconds, i.e. here 1s.
- The `end` details the timestamp at the last value in seconds i.e. here 4s.
- The `number` is the number of points in the spectrum i.e. here 5.
- The `speed` is the speed of the machine, or reference frequency, in Hz i.e. here 2Hz
- The `data` field is a comma separated list of points, wrapped in escaped quote marks. The data is interpreted as a list of 64-bit floating point values. The number of these must match the number parameter.
- Numbers must use a period (`.`) as the decimal separator.
- Each row must have the same number of columns as the header row.

## File

### Name

File names should be made up of two parts: the mapping ID for the sensor the data is for and a unique value to avoid collisions, separated by `@`. As an example: `motorM32@1651149458.csv`.

The initial section of the filename is used to route the data to the correct sensor based on the mapping ID. If a new mapping ID/filename is provided, Senseye will provision a new sensor associated with this mapping ID and route all future files using this mapping ID as the filename to the sensor. In the above example, the mapping ID would be `motorM32`.

The later section is included to avoid file collisions. We recommend using a unix timestamp, but any unique value is suitable. In the above example, the unix timestamp is `1651149458`.

### Size

Please limit CSV files to no more than 10MB each.

### Encoding

The data file must use UTF-8 character encoding.

## Structure

The JSON format for waveforms is as follows:

```json
[
    {
        "mappingId": "mappingId",
        "timestamp": "2024-03-04T18:13:00Z",
        "amplitudeUnit": "mm/s",
        "spacing": 1.0,
        "end": 4.0,
        "number": 5,
        "data": [
            0,
            1,
            2,
            3,
            4
        ],
        "speed": 1.0,
        "measurementType": "acceleration"
        |
        "velocity"
    }
]

```

```json
[{
    "sensorId":         "sensorId",
    "timestamp":        "2024-03-04T18:13:00Z",
    "amplitudeUnit":    "mm/s",
    "spacing":          1.0,
    "end":              4.0,
    "number":           5,
    "data":             [0, 1, 2, 3, 4],
    "speed":            1.0,
    "measurementType":  "acceleration" | "velocity"
}]

```

- The `mappingId` is a unique mapping ID for a sensor. If a sensor with the mapping ID does not exist, a new sensor will be provisioned.
- The `sensorId` is a UUID that Senseye has allocated to a sensor. This field is used only if the sensor already exists.
- The `timestamp` should be in RFC3339 dates and times, including timezone - more details on the required format are available [here](../../../../introduction/timeanddates.html).
- It is assumed that the data starts at 0 Hz.
- The `amplitudeUnit` should be a string value representing the units of measurement for the amplitude. For velocity, 'in/s', 'mm/s' and 'm/s' are valid units. For acceleration, 'in/s^2', 'mm/s^2' and 'm/s^2' are valid units. Conversion must be undertaken to one of these units prior to sending data to Senseye.
- The `spacing` details the gap between bins in seconds i.e. here 1s
- The `end` details the frequency at the last value in seconds i.e. here 4s.
- The `number` is the number of points in the spectrum i.e. here 5.
- The `speed` is the speed of the machine, or reference frequency, in Hz i.e. here 2Hz
- The `data` field is a comma separated list of points, wrapped in escaped quote marks. The data is interpreted as a list of 64-bit floating point values. The number of these must match the number parameter.

## File

### Name

To avoid file collisions, the name of the file should be unique. To aid debugging, we recommend composing the file name using two components: an ID for the asset or group of assets being monitored, and a unix timestamp, separated by `@` - as an example: `motors@1651149458.json`.

### Size

Please limit the JSON request body to no more than 10MB.

Senseye can connect directly to your historian to gather time series data. We support a range of historians out of the box and can develop custom integrations as required.

Your historian must be made accessible to Senseye. In terms of networking, we use a single egress IP address which can be whitelisted within firewalls. If required, we can support VPNs. We recommend that encryption is used during transit and a trusted certificate is applied to the historian. Furthermore, we require suitable authentication credentials and the correct level of access to the data within the historian.

When connecting to a historian, we do not copy the raw data. We will access the data in two situations:

1. The charts in the application show the raw data rather than aggregated data. In this situation, Senseye will request the raw data from your historian, as such, the historian's speed will have a direct impact on the performance of the Senseye web interface.
1. Calculating aggregations. Senseye will periodically check your historian for the presence of new data. When new data is found, hourly and daily aggregations will be calculated which are then stored within Senseye.

Please contact [support](mailto:senseye.support.industry@siemens.com) for more information on this topic.

Senseye makes available an Insights Hub application which provides access to Senseye's application within Insights Hub, allows Senseye to access data stored within Insights Hub's time series database, and optionally allows Senseye to write case information to Asset Health and Maintenance. The application is not available on the Insights Hub marketplace, but can be provisioned by Senseye support.

Info

Note that Senseye does not share authentication with Insights Hub. Users will need both an Insights Hub account, and a Senseye account. If users require a Senseye account, [invitations to join Senseye can be sent by a Senseye administrator](https://help.senseye.io/hc/en-gb/articles/360017643179-Invite-new-users).

To access the Senseye application in Insights Hub users will need to log into Insights Hub, and then log into Senseye upon opening the Senseye application.

## Provisioning

To request for the application to be provisioned to your Insights Hub tenant please send the below information in an email to [support](mailto:senseye.support.industry@siemens.com):

- Insights Hub Tenant
  - For example, abc in the Insights Hub URL `https://abc.eu1.mindsphere.io/`
- Tenant Administrator Email Address
  - This is the email address for a tenant administrator with permissions to provision applications
- Senseye Organization Name or ID
  - The name of your organization in Senseye or it's ID

A support member will then start the provisioning process:

1. A provisioning request will be sent to tenant administrators requesting permission to provision the application
1. A tenant administrator accepts the request, automatically starting deployment
1. Tenant administrators will receive a confirmation email and notification in Insights Hub that the application has been deployed
1. Confirm to the support member that the application has been deployed
1. Tenant administrators can start giving Insight Hub users the Senseye role
   - Go to the Insight Hub Settings
   - Head to the roles page, and filter for senseye
   - Add the role to users / user groups required
1. Senseye support will confirm that setup is complete

## Time Series Data

Data in Insights Hub can be brought into Senseye by mapping asset and aspect pairs in the Insight Hub data model to [sensors](../../introduction/terminology.html#sensor) in Senseye. Before starting, you will need the ID of the asset and the name of the aspect in Insights Hub.

Instructions for creating the sensor can be found under the [Create a new Sensor on an Asset](https://help.senseye.io/hc/en-gb/articles/360017663160-Manage-the-Hierarchy-Sensors) help page. For Insights Hub specifically:

1. Chose your Insights Hub tenant from the Sensor Hub dropdown
1. Enter the Asset ID and Aspect Name
   - This should be entered in the format of "AssetID:AspectName"
   - For example, if the AssetID is `85058830de464111b97407fed181155c` and the aspect is `msdata1` you should enter `85058830de464111b97407fed181155c:msdata1`

## Case Notifications

Senseye is compatible with [Asset Health and Maintenance](https://documentation.mindsphere.io/MindSphere/apps/insights-hub-asset-health-and-maintenance/overview.html) (AHM). AHM can act as an aggregation layer over multiple condition monitoring systems, before outputting to a asset management system. Senseye cases can optionally be sent to AHM for use in other integrations. To request this feature to be turned on please contact [support](mailto:senseye.support.industry@siemens.com).

Senseye has an internal time series database, often called the internal hub or iHub, that customers may store data in for the purposes of predictive maintenance. A variety of methods are available for pushing data to be stored by Senseye, enabling choice in data integration method.

Depending on the push method chosen, you may identify [sensors](../../introduction/terminology.html#sensor) by either an internal Senseye ID (in the form of a [UUID](../../introduction/terminology.html#uuid)), or an external reference, which will be referred to as mapping IDs from here on out.

For all new integrations we strongly recommend using mapping IDs as this enables Senseye to automatically provision new [sensors](../../introduction/terminology.html#sensor) for you. When data is sent to a new mapping ID Senseye will automatically create a new [sensor](../../introduction/terminology.html#sensor) associated with this mapping ID and make it available for import into the [hierarchy](../../introduction/terminology.html#hierarchy).

If internal Senseye IDs are used, then prior to pushing data for a new sensor it is necessary to create the required internal hub [sensor](../../introduction/terminology.html#sensor) and record it's internal ID. The steps for creating sensors can be found [here](../sensor.html).

Info

Mapping IDs are stored against the [sensor](../../introduction/terminology.html#sensor) as an [external ID](../../introduction/terminology.html#external-id) of type `SENSOR`. Please note that any mapping IDs used must be unique across your organization for the purposes of sending data to Senseye (this may include sites in other locations).

We therefore recommend creating a sensible naming convention for your use case.

If you have any queries about these methods, or using our internal time series database in general, please contact [support](mailto:senseye.support.industry@siemens.com).

Machine data can be sent via [AWS S3](https://aws.amazon.com/s3/), [Azure Blob Storage](https://azure.microsoft.com/en-gb/services/storage/blobs/) and S3 compatible data stores such as [Oracle Object Storage](https://docs.oracle.com/en-us/iaas/Content/Object/Concepts/objectstorageoverview.htm). Senseye will periodically gather the files and store the time series data within Senseye's time series database.

Data should be sent in our one of our standard formats. For time series data, these are [CSV](../formats/timeseries/csv.html), [JSON](../formats/timeseries/json.html), or [Parquet](../formats/timeseries/parquet.html). For vibration data, these are [Waveform CSV/JSON](../formats/vibration/waveforms/_index.html) or [Spectrum CSV/JSON](../formats/vibration/spectra/_index.html). See their respective pages for more information on the structure we expect. We can also support custom data formats - please contact [support](mailto:senseye.support.industry@siemens.com) for more information.

Note

Once a file has been processed, it will be deleted from the data store to avoid reprocessing

## AWS S3

Senseye can retrieve data files from your bucket, or we can create one for your use. If Senseye is to connect to your bucket, we require the following information:

- Account access and secret key, with read and delete permissions on the bucket's contents
- Bucket name

Files should be placed at the root of the bucket. As noted above, we delete files once they have been processed, depending on the data volume, you may wish to activate versioning on the bucket so that files can be recovered if required.

For S3 compatible data stores, the only additional requirement is to provide an endpoint URL for the data store.

## Azure Blob

Like S3, we can retrieve data files from your storage account, or we can create one for your use. If Senseye is to connect to your storage account, we require the following information:

- Storage account name
- Container name
- Access key associated with the storage account

Files can be placed within a directory structure if desired, but it is not required.

If required, Senseye can accept time series data via email.

Data should be sent in our one of our standard formats. For time series data, these are [CSV](../formats/timeseries/csv.html), [JSON](../formats/timeseries/json.html), or [Parquet](../formats/timeseries/parquet.html). See their respective pages for more information on the structure we expect. We can also support custom data formats - please contact [support](mailto:senseye.support.industry@siemens.com) for more information. To help reduce the attachment size, it is possible to combine one or more files into a zip archive and send this.

To configure this integration, Senseye needs to know the email address which will be used to send the data. Once defined, we will provide an email address which will receive the data.

Note

Currently, only time series data can be sent via email; vibration data is not yet supported.

[MQTT](https://en.wikipedia.org/wiki/MQTT) is a lightweight, publish-subscribe, machine to machine network protocol. Machine data can be sent via MQTT to Senseye's broker. Senseye will consume messages and store the time series within Senseye's time series database.

## MQTT Message

A message comprises of a topic and a payload. The topic is used to describe the client and data format of the payload, the payload contains the machine data.

Senseye has designed topics that are composed of the following terms, each term is delimited by `/`.

1. Version, currently `v1`
1. Organization ID. This will be provided by Senseye during onboarding and will typically be prefixed `group|...`
1. Data format, e.g. `csv`
1. Payload ID. An identifier for the source of the payload.
   - For formats where Senseye sensor IDs or mappings IDs aren't specified in the format, for example CSV, the payload ID will be used as a mapping ID.
   - For format where Senseye sensor IDs or mapping IDs are specified in the format, for example JSON, the payload ID is not used for data routing but should still be provided.
1. (Optional) The measure name that the time series belongs to. This is only needed if the payload containing the time series data does not include the measure name.

An example of a `topic` where the measure name is included in the payload and so is not needed in the topic is `v1/group|abc/json/payload-id`.

An example of a `topic` where the measure name is not included in the payload and is therefore needed explicitly in the topic is `v1/group|abc/csv/payload-id/a-measure-name`.

Data should be sent in our one of our standard formats. For time series data, these are [CSV](../formats/timeseries/csv.html), [JSON](../formats/timeseries/json.html), or [Parquet](../formats/timeseries/parquet.html). For vibration data, these are [Waveform CSV/JSON](../formats/vibration/waveforms/_index.html) or [Spectrum CSV/JSON](../formats/vibration/spectra/_index.html). See their respective pages for more information on the structure we expect.

We can also support custom data formats - please contact [support](mailto:senseye.support.industry@siemens.com) for more information.

## Connecting to Senseye's MQTT Broker

Senseye's MQTT broker is hosted at `mqtt.senseye.io` on port `8883`. Connections to the broker are over TLS.

To use the broker, clients will need Senseye to generate a `username` and `password`. These can be obtained from [support](mailto:senseye.support.industry@siemens.com). These can be used by all devices publishing MQTT data.

Under the MQTT protocol, when messages are published to the broker, a `ClientID` is required. This is a unique identifier between every sending device or computer that publishes MQTT data and the broker.

An example. Suppose a client has two physical devices, each has MQTT publish capability, and there is a desire for the data from the two devices to be aggregated together at Senseye under a single external-id. Each device will need to use their own unique `ClientID`s to allow both devices to connect to the broker in parallel. But, both devices could use the same topic (i.e. provide the same external ID) so that Senseye treats the two physical devices as one logical device.

The MQTT protocol allows for a Quality of Service (QoS) measure to be provided. Currently Senseye's broker supports `At most once` – the message is sent only once and the client and broker take no additional steps to acknowledge delivery (fire and forget).

For MQTT v3.1 and v3.1.1 clients there is no mechanism for the Senseye broker to provide a `keep-alive` value. For these clients please ensure a `keep-alive` parameter is provided and that it is non-zero, our recommendation is 60s (our broker's default). If an invalid `keep-alive` is provided by the client a CONNACK message with the "identifier rejection" reason code will be sent, and the client disconnected.

[WebDAV](https://en.wikipedia.org/wiki/WebDAV) is a set of extensions to HTTP, which allows user agents to collaboratively author contents directly in an HTTP web server by providing facilities for concurrency control and namespace operations. Time series data can be sent via a WebDAV client to Senseye’s WebDAV endpoint (`https://webdav.senseye.io`). Senseye will consume the files and store the data within Senseye’s time series database.

## Connecting to Senseye's WebDAV Endpoint

Senseye's WebDAV Endpoint is hosted at `webdav.senseye.io`. Connections to the endpoint are over TLS.

To use the endpoint, clients will need Senseye to generate a `username` and `password`.

## WebDAV

Senseye's WebDAV Endpoint supports multi-tenancy, so when Senseye sets up your tenant, they'll be able to give you a specific url to direct your WebDAV requests to, for example: `webdav.senseye.io/webdav/a-tenant-name`

NOTE: There will be one tenant set up per data format, for example: if tenant-1 wanted to send csv's and json, you'd have to ask for two tenants to be set up:

1. csv - `webdav.senseye.io/webdav/tenant-1-csv`
1. json - `webdav.senseye.io/webdav/tenant-1-json`

Data should be sent in our one of our standard formats. For time series data, these are [CSV](../formats/timeseries/csv.html), [JSON](../formats/timeseries/json.html), or [Parquet](../formats/timeseries/parquet.html). For vibration data, these are [Waveform CSV/JSON](../formats/vibration/waveforms/_index.html) or [Spectrum CSV/JSON](../formats/vibration/spectra/_index.html). See their respective pages for more information on the structure we expect. We can also support custom data formats - please contact [support](mailto:senseye.support.industry@siemens.com) for more information.

# Push to Senseye API

Senseye provides multiple APIs for pushing data to the platform. This section provides detailed information on the available APIs for time series and vibration data.

Time series data can be provided by sending data to HTTP APIs, either GraphQL, CSV REST, or JSON REST. When sending data either a Mapping ID or a Senseye sensor ID (in the form of a UUID) can be used to address a sensor. If a Mapping ID is used, and a sensor does not already exist with this mapping ID, a new sensor is automatically provisioned.

Info

For user tokens, the associated user needs to be a system administrator to send data.

Machine tokens require permissions to write data and create sensors. If using Mapping IDs with a machine token, your token will also require permissions to read and write mapping IDs.

## GraphQL

We support sending data via [GraphQL](../../../introduction/graphql.html) using the [storeSensorMeasureData mutation](https://graphqldocs.senseye.io/#mutation-storeSensorMeasureData).

Query

```graphql
mutation storeSensorMeasureData($input: StoreSensorMeasureDataInput!) {
    storeSensorMeasureData(input: $input) {
        clientMutationId
    }
}

```

Variables

```json
{
    "input": {
        "mappingId": "mappingId",
        "dataPoints": [{
            "measureName": "temperature",
            "floatValue": 66,
            "timestamp": "2022-04-29T12:55:14+01:00"
        }, {
            "measureName": "current",
            "floatValue": 525,
            "timestamp": "2022-04-29T12:55:14+01:00"
        }, {
            "measureName": "recipe",
            "stringValue": "87N",
            "timestamp": "2022-04-29T12:50:00+01:00"
        }, {
            "measureName": "run",
            "booleanValue": true,
            "timestamp": "2022-04-29T12:50:00+01:00"
        }],
        "clientMutationId": "Your Identifier"
    }
}

```

Response

```json
{
    "data": {
        "clientMutationId": "Your Identifier"
    }
}

```

Query

```graphql
mutation storeSensorMeasureData($input: StoreSensorMeasureDataInput!) {
    storeSensorMeasureData(input: $input) {
        clientMutationId
    }
}

```

Variables

```json
{
    "input": {
        "sensorId": "sensorId",
        "dataPoints": [{
            "measureName": "temperature",
            "floatValue": 66,
            "timestamp": "2022-04-29T12:55:14+01:00"
        }, {
            "measureName": "current",
            "floatValue": 525,
            "timestamp": "2022-04-29T12:55:14+01:00"
        }, {
            "measureName": "recipe",
            "stringValue": "87N",
            "timestamp": "2022-04-29T12:50:00+01:00"
        }],
        "clientMutationId": "Your Identifier"
    }
}

```

Response

```json
{
    "data": {
        "clientMutationId": "Your Identifier"
    }
}

```

The `clientMutationId` response in the above example can be ignored. Up to 15000 datapoints can be stored in a single storeSensorMeasureData mutation.

## CSV REST

The CSV formatted data should conform to the structure detailed [here](../../formats/timeseries/csv.html). It should be sent within the body of a POST request as shown below:

```http
POST /v1/hub/mappings/{mappingId}/data HTTP/1.1
Host: api.senseye.io
Authorization: Bearer your_access_token
Content-Type: text/csv

DATETIME,temperature,current,product,run
2022-04-29T11:13:14+01:00,60,456,"A24",true
2022-04-29T12:01:45+01:00,61,500,"A24",false

```

```http
POST /v1/hub/sensors/{senseyeId}/data HTTP/1.1
Host: api.senseye.io
Authorization: Bearer your_access_token
Content-Type: text/csv

DATETIME,temperature,current,product,run
2022-04-29T11:13:14+01:00,60,456,"A24",true
2022-04-29T12:01:45+01:00,61,500,"A24",false

```

## JSON REST

We use [SenML](https://datatracker.ietf.org/doc/html/rfc8428) to transfer sensor data via REST; this is a popular JSON format. As an example:

SenML

```json
{
    "e": [
        {
            "n": "temperature",
            "v": 65,
            "t": 1651236101
        },
        {
            "n": "current",
            "v": 520,
            "t": 1651236101
        },
        {
            "n": "product",
            "sv": "A24",
            "t": 1651236101
        },
        {
            "n": "run",
            "bv": true,
            "t": 1651236101
        }
    ]
}

```

Where `n` is the measure name, `t` is the unix time (see [here](../../../introduction/timeanddates.html) for more info), `v` is a numeric value, `sv` is a string value and `bv` is a boolean value.

A SenML representation of your data should be posted as follows:

```http
POST /v1/hub/mappings/{mappingId}/data HTTP/1.1
Host: api.senseye.io
Authorization: Bearer your_access_token
Content-Type: application/json

{
    "e": [{
        "n": "temperature",
        "v": 65,
        "t": 1651236101
    }, {
        "n": "current",
        "v": 520,
        "t": 1651236101
    }, {
        "n": "product",
        "sv": "A24",
        "t": 1651236101
    }, {
        "n": "run",
        "bv": true,
        "t": 1651236101
    }]
}

```

```http
POST /v1/hub/sensors/{sensorId}/data HTTP/1.1
Host: api.senseye.io
Authorization: Bearer your_access_token
Content-Type: application/json

{
    "e": [{
        "n": "temperature",
        "v": 65,
        "t": 1651236101
    }, {
        "n": "current",
        "v": 520,
        "t": 1651236101
    }, {
        "n": "product",
        "sv": "A24",
        "t": 1651236101
    }, {
        "n": "run",
        "bv": true,
        "t": 1651236101
    }]
}

```

Senseye has support for ingesting raw vibration waveforms and spectra. Currently, we can handle either CSV or JSON in our own formats, or we additionally have built in support for exports from third-party software. If your export format is not supported, please contact [support](mailto:senseye.support.industry@siemens.com) for more information.

## Processing

When Senseye receives a velocity waveform, we compute the acceleration waveform from it, and similarly if we receive an acceleration waveform we compute the velocity waveform. This allows us to compute a number of condition indicators for both acceleration and velocity signals:

- Root Mean Square (RMS)
- Peak to Peak (P2P)
- Zero to Peak (Z2P)
- Standard Deviation
- Kurtosis
- Skew
- Crest Factor
- Shape Factor

If a spectrum is sent rather than a raw waveform, then we simply use this (note that acceleration spectra are converted to velocity spectra), but note that the above condition indicators are not derived from it. From the velocity waveform (either processed by us, or received directly) we then compute directly the spectrum using a Hanning window. We attempt to extract the amplitude at key frequencies as measures, which are then trackable over time as condition indicators of your machinery. Currently, we will extract the amplitude matching the rotational speed of your machine and its harmonics, which we label as `v_1x` - `v_5x`. At present, the rotational speed at the time the measurement was taken (or the average speed for a variable drive machine) needs to be sent in Hz along with each sample so that we can perform this calculation, even if the machine is set to work at a fixed rotational speed. This speed is created as an additional measure and is automatically set up as a control parameter for the asset, with the name `rotational-speed`. We retain the velocity spectrum for visualization in the Senseye app; by clicking a point on a measure chart set to 'raw' mode, the spectra from which the measure was extracted can be viewed. Please note that you should not attempt to send multiple corresponding data for the same timestamp - please send only one of velocity/acceleration waveform or spectra. If you have the waveform data, this should be preferred.

When sending data either a Mapping ID or a Senseye sensor ID (in the form of a UUID) can be used to address a sensor. If a Mapping ID is used, and a sensor does not already exist connected with this mapping ID, a new sensor is automatically provisioned. A sensor can alternatively be created manually as detailed [here](../../sensor.html).

Info

For user tokens, the associated user needs to be a system administrator to send data.

Machine tokens require permissions to write data and create sensors. If using Mapping IDs with a machine token, your token will also require permissions to read and write mapping IDs.

### JSON REST

#### Waveforms

The JSON formatted data should conform to the structure detailed [here](../../formats/vibration/waveforms/json.html). It should be sent within the body of a POST request as shown below:

```http
POST /v1/hub/mappings/{mappingId}/waveform_data HTTP/1.1
Host: api.senseye.io
Authorization: Bearer your_access_token
Content-Type: application/json

[
    {
        "timestamp":       "2024-03-04T18:13:00Z",
        "amplitudeUnit":   "m/s" | "m/s^2" | "mm/s" | "mm/s^2 | "in/s" | "in/s^2",
        "spacing":         1.0,
        "end":             4.0,
        "number":          5,
        "data":            [0, 1, 2, 3, 4],
        "speed":           1.0,
        "measurementType": "velocity" | "acceleration"
    },
]

```

```http
POST /v1/hub/sensors/{senseyeId}/waveform_data HTTP/1.1
Host: api.senseye.io
Authorization: Bearer your_access_token
Content-Type: application/json

[
    {
        "timestamp":       "2024-03-04T18:13:00Z",
        "amplitudeUnit":   "m/s" | "m/s^2" | "mm/s" | "mm/s^2 | "in/s" | "in/s^2",
        "spacing":         1.0,
        "end":             4.0,
        "number":          5,
        "data":            [0, 1, 2, 3, 4],
        "speed":           1.0,
        "measurementType": "velocity" | "acceleration"
    },
]

```

Please note that waveform time unit (and thus `spacing` and `end`) must be in units of seconds.

#### Spectra

The JSON formatted data should conform to the structure detailed [here](../../formats/vibration/spectra/json.html). It should be sent within the body of a POST request as shown below:

```http
POST /v1/hub/mappings/{mappingId}/freq_data HTTP/1.1
Host: api.senseye.io
Authorization: Bearer your_access_token
Content-Type: application/json

[
    {
        "timestamp":       "2024-03-04T18:13:00Z",
        "amplitudeUnit":   "m/s" | "m/s^2" | "mm/s" | "mm/s^2 | "in/s" | "in/s^2",",
        "spacing":         1.0,
        "end":             4.0,
        "number":          5,
        "data":            [0, 1, 2, 3, 4],
        "speed":           1.0,
        "measurementType": "velocity" | "acceleration"
    },
]

```

```http
POST /v1/hub/sensors/{senseyeId}/freq_data HTTP/1.1
Host: api.senseye.io
Authorization: Bearer your_access_token
Content-Type: application/json

[
    {
        "timestamp":       "2024-03-04T18:13:00Z",
        "amplitudeUnit":   "m/s" | "m/s^2" | "mm/s" | "mm/s^2 | "in/s" | "in/s^2",,
        "spacing":         1.0,
        "end":             4.0,
        "number":          5,
        "data":            [0, 1, 2, 3, 4],
        "speed":           1.0,
        "measurementType": "velocity" | "acceleration"
    },
]

```

Please note that spectra frequency units (and thus `spacing` and `end`) must be in Hz.

### CSV REST

#### Waveforms

The CSV formatted data format is detailed [here](../../formats/vibration/waveforms/csv.html). It can be sent to Senseye as a POST request:

```http
POST /v1/hub/mappings/{mappingId}/waveform_data HTTP/1.1
Host: api.senseye.io
Authorization: Bearer your_access_token
Content-Type: text/csv

timestamp,amplitudeUnit,measurementType,spacing,end,number,speed,data
2024-03-04T18:13:00Z,mm/s,velocity,1,4,5,2,"0,1,2,3,4"

```

```http
POST /v1/hub/sensors/{sensorId}/waveform_data HTTP/1.1
Host: api.senseye.io
Authorization: Bearer your_access_token
Content-Type: text/csv

timestamp,amplitudeUnit,measurementType,spacing,end,number,speed,data
2024-03-04T18:13:00Z,mm/s,velocity,1,4,5,2,"0,1,2,3,4"

```

#### Spectra

The CSV formatted data should conform to the structure detailed [here](../../formats/vibration/spectra/csv.html). It should be sent within the body of a POST request as shown below:

```http
POST /v1/hub/mappings/{mappingId}/freq_data HTTP/1.1
Host: api.senseye.io
Authorization: Bearer your_access_token
Content-Type: text/csv

timestamp,amplitudeUnit,measurementType,spacing,end,number,speed,data
2024-03-04T18:13:00Z,mm/s,velocity,1,4,5,2,"0,1,2,3,4"

```

```http
POST /v1/hub/sensors/{senseyeId}/freq_data HTTP/1.1
Host: api.senseye.io
Authorization: Bearer your_access_token
Content-Type: text/csv

timestamp,amplitudeUnit,measurementType,spacing,end,number,speed,data
2024-03-04T18:13:00Z,mm/s,velocity,1,4,5,2,"0,1,2,3,4"

```

### SKF @ptitude Analyst CSV

The CSV formatted data should conform to the structure detailed [here](../../formats/vibration/spectra/skf.html). It should be sent within the body of a POST request as shown below:

```http
POST /v1/hub/mappings/{mappingId}/freq_data/skf HTTP/1.1
Host: api.senseye.io
Authorization: Bearer your_access_token
Content-Type: text/csv

"Point Path","DTS","Unit","Detection","Channel","Lines","End Frequency","Speed (Hz)","Process Value","Unit","BOV / GAP","DIGITAL","Data"
"NodeID","2023-02-21 9:42:39 AM","mm/sec","Peak","1","5","4","2","N/A","N/A","N/A","N/A","1","2","3","4","5"

```

```http
POST /v1/hub/sensors/{senseyeId}/freq_data/skf HTTP/1.1
Host: api.senseye.io
Authorization: Bearer your_access_token
Content-Type: text/csv

"Point Path","DTS","Unit","Detection","Channel","Lines","End Frequency","Speed (Hz)","Process Value","Unit","BOV / GAP","DIGITAL","Data"
"NodeID","2023-02-21 9:42:39 AM","mm/sec","Peak","1","5","4","2","N/A","N/A","N/A","N/A","1","2","3","4","5"

```

Note that we accept only velocity spectra exported from SKF. We do not currently process exported files with a mixture of acceleration and velocity measurements.

# Notifications

A Case within the Senseye PdM application is the prime source of notification to the user. It is there to bring attention to an asset and contains evidence to allow the user to understand the reason for the case generation.

Senseye PdM can be integrated with maintenance systems such that Senseye Cases are automatically converted into a Notification or Work Request. In doing so, maintainers will have a frictionless means to triage those assets that warrant attention.

Manual work notifications is a feature that allows a case to be flagged for notification to a connected CMMS. When enabled, cases have a Trigger Manual CMMS option. See [trigger CMMS](https://help.senseye.io/hc/en-gb/articles/8480342253724-How-to-manually-trigger-a-work-notification-in-a-CMMS) for more details including how to enable it.

Where a *push* integration is in place, it can be configured to push a notification to a CMMS system only when Trigger Manual Work Notification is clicked.

Where a *pull* integration is being used via Senseye's api, then the following GraphQL query can be used to determine whether a case has a Manual Notification Event:

Query

```graphql
query($id: ID!,$offset: Int, $limit: Int) {
    sublevel(id:$id){
        cases(offset: $offset, limit: $limit){
            results {
                asset{
                    name
                    id
                }
                latestManualNotificationEvent {
                    eventTime
                    state
                    message
                    user{
                        email
                        name

                    }
                }
            }

        }
    }
}

```

Variables

```json
{
    "offset": 0,
    "limit": 50,
    "lastPollTime": "2025-01-03T00:00:00+01:00"
}

```

Response

```json
{
    "data": {
        "sublevel": {
            "cases": {
                "results": [
                    {
                        "asset": {
                            "name": "Main Water Pump .,.",
                            "id": "6e480890-d9cd-4a63-9037-6ed87f51c3fa"
                        },
                        "latestManualNotificationEvent": null
                    },
                    {
                        "asset": {
                            "name": "Air Compressor DG 2.14",
                            "id": "cf4dc3bc-815e-4e5a-a213-bc7f96017000"
                        },
                        "latestManualNotificationEvent": {
                            "eventTime": "2025-02-19T09:06:31Z",
                            "state": "ENABLED",
                            "message": null,
                            "user": {
                                "email": "anonymizedy@senseye.io",
                                "name": "anonymized"
                            }
                        }
                    },
                    {
                        "asset": {
                            "name": "Main Water Pump .,.",
                            "id": "6e480890-d9cd-4a63-9037-6ed87f51c3fa"
                        },
                        "latestManualNotificationEvent": {
                            "eventTime": "2024-11-22T17:24:55Z",
                            "state": "ENABLED",
                            "message": "Bearing Change",
                            "user": {
                                "email": "clark.c.green.anonymized@senseye.io",
                                "name": "Clark C. Green"
                            }
                        }
                    },
                    {
                        "asset": {
                            "name": "Air Compressor DG 2.14",
                            "id": "cf4dc3bc-815e-4e5a-a213-bc7f96017000"
                        },
                        "latestManualNotificationEvent": null
                    },
                ]  
            }
        }
    }
}

```

Where the CMMS trigger information is included in the last section.

With GraphQL, a script can be created to first query to retrieve all open cases, then pass each case ID through to retrieve any Manual Notification Events.

Senseye PdM has an extensive [GraphQL API](../introduction/graphql.html) that can be used to poll for new cases. Alongside [External ID](../introduction/externalids.html) support, it is often selected as a low friction method for notification integration.

## Query

The following request makes use of the [cases query on an organization](https://graphqldocs.senseye.io/#definition-Organization) to gather Cases which have opened since a given date. This can be used to gather the new cases since the last time the API was polled.

[Cases](https://graphqldocs.senseye.io/#definition-CaseFile) have many attributes which may be utilized within the integration, the below example provides general information about the case along with the name and External ID for the associated asset.

The information generated by GenAI can be retrieved using the description value in the query.

The description field can be divided into:

- **What’s happening** – An explanation of what caused the case to open based on a given measure and the time the upward/downward trend or anomaly started.
- **Possible explanations** - Describes three possible explanations for the case to be opened. The three main reasons are marked with "\*\*" followed by a description what could have caused that type of failure.

The links in the text will redirect you to measures and sensor information correlated to the case.

The ID for the chosen level of the hierarchy can be taken by extracting it from the URL in the application when the user is at the chosen level of the hierarchy. The ID being the string between `nodeid=` and the next `&`. For example, if the URL is `https://app.senseye.io/settings/hierarchy-management?nodeId=9436480a-595d-4656-9e1c-483a84070d40&activeTab=properties&activeNodeConfigurationTab=computed` then the ID would be `9436480a-595d-4656-9e1c-483a84070d40`.

Query

```graphql
query($offset: Int, $limit: Int, $lastPollTime: Time) {
    organization {
        cases(offset: $offset, limit: $limit, openedAfter: $lastPollTime) {
            results {
                id
                acknowledgement {
                    timestamp
                }
                latestManualNotificationEvent{
                    eventTime
                    message
                    state
                    user{
                        id
                        name
                        email
                    }
                }
                asset {
                    name
                    externalId
                }
                isOpen
                openedAt
                closedAt
                primaryTrigger
                triggers
                description
            }

        }
    }
}

```

Variables

```json
{
    "offset": 0,
    "limit": 50,
    "lastPollTime": "2025-01-03T00:00:00+01:00"
}

```

Response

```json
{
    "data": {
        "organization": {
            "cases": {
                "results": [
                    {
                        "id": "1cf5a1c0-152b-4e3b-bbb8-c43e74088874",
                        "acknowledgement": null,
                        "latestManualNotificationEvent": null,
                        "asset": {
                            "name": "Coating Drum CP5.1",
                            "externalId": null
                        },
                        "isOpen": true,
                        "openedAt": "2025-03-04T09:00:00Z",
                        "closedAt": null,
                        "primaryTrigger": "TREND",
                        "triggers": [
                            "TREND"
                        ],
                        "description": "**What's happening**\n\n- Falling trend on https://app.senseye.io/measure/measure:e3c32b0f-b5ee-4f20-9519-9af90b9acffc:dm1 started 23 days ago.\n- Rising trend on https://app.senseye.io/measure/measure:e3c32b0f-b5ee-4f20-9519-9af90b9acffc:dm2 started 21 days ago.\n- Correlated behaviour on https://app.senseye.io/measure/measure:e3c32b0f-b5ee-4f20-9519-9af90b9acffc:dm3 and https://app.senseye.io/measure/measure:e3c32b0f-b5ee-4f20-9519-9af90b9acffc:dm4.\n**Possible explanations**\n\n- **Bearing wear or misalignment.** The rising trend in D.E. Bearing RMS and correlated behavior in NDE Bearing RMS suggest increased vibration, which could indicate bearing wear or misalignment. This is a common issue in rotating machinery and should be investigated to prevent further damage.\n\n  - https://app.senseye.io/case/8c36ad73-d143-4490-8972-6e933b246532\n  - https://app.senseye.io/case/b0fb16f7-89ca-447c-8bea-f32abb6373e6\n \n- **Electrical issues or motor inefficiency.** A falling trend in Motor Current Max could indicate reduced load or efficiency issues with the motor. This might be due to electrical problems or changes in operational conditions.\n\n \n- **Normal operational adjustments.** The absence of recent maintenance or notes might suggest that the observed trends are due to normal operational adjustments. However, given the concerning trends in bearing RMS values, this should be verified.\n\n "
                    },
                    {
                        "id": "74d228a3-5ed9-418f-82e6-6c76155df827",
                        "acknowledgement": null,
                        "latestManualNotificationEvent": null,
                        "asset": {
                            "name": "Outbound Pallet Conveyor AN 3",
                            "externalId": null
                        },
                        "isOpen": true,
                        "openedAt": "2025-03-04T09:00:00Z",
                        "closedAt": null,
                        "primaryTrigger": "ANOMALY",
                        "triggers": [
                            "ANOMALY"
                        ],
                        "description": "**What's happening**\n\n- Upward level shift on measure https://app.senseye.io/measure/measure:d7264cf8-2873-43cc-9c2a-b535eba7a906:field4 on sensor https://app.senseye.io/sensor/d7264cf8-2873-43cc-9c2a-b535eba7a906.\n- Upward level shift on measure https://app.senseye.io/measure/measure:d7264cf8-2873-43cc-9c2a-b535eba7a906:field5 on sensor https://app.senseye.io/sensor/d7264cf8-2873-43cc-9c2a-b535eba7a906.\n- Upward level shift on measure https://app.senseye.io/measure/measure:d7264cf8-2873-43cc-9c2a-b535eba7a906:field1 on sensor https://app.senseye.io/sensor/d7264cf8-2873-43cc-9c2a-b535eba7a906.\n**Possible explanations**\n\n- **Gearbox or motor misalignment.** The upward step in gearbox RMS acceleration and motor RMS velocity could indicate misalignment issues. This simultaneous change suggests a mechanical issue affecting both components, which might require alignment checks.\n\n \n- **Increased load or tension.** The upward step in torque and belt tension could be due to increased load on the conveyor. This aligns with scenarios where operational load increases, affecting the torque and tension measures.\n\n \n- **Normal operational variation.** Conveyors can experience normal variations in torque and RMS measures due to operational changes. However, the simultaneous upward step in multiple measures is unusual for normal variation, suggesting a review of application sensitivity might be needed.\n\n "
                    }
                ]
            }
        }
    }
}

```

Handling different maintenance systems across sites...

If different maintenance systems are used in different sites of your organization, you may wish to query just for cases within a given site. This is possible by querying for a specific sub-level of the hierarchy, then gathering the cases just from that branch of the hierarchy.

As example, the following section of the above query:

Query

```graphql
query($offset: Int, $limit: Int, $lastPollTime: Time) {
    organization {
        cases(offset: $offset, limit: $limit, openedAfter: $lastPollTime) {
            ...
        }
    }
}

```

could be replaced with the [sublevel query](https://graphqldocs.senseye.io/#query-sublevel):

Query

```graphql
query($offset: Int, $limit: Int, $lastPollTime: Time, $sublevelGUID: String) {
    sublevel(id: $sublevelGUID) {
        cases(offset: $offset, limit: $limit, openedAfter: $lastPollTime) {
            ...
        }
    }
}

```

or External IDs could be used to identify the sub-level using [hierarchyNodeByExternalID](https://graphqldocs.senseye.io/#query-hierarchyNodeByExternalID) and suitable type casting:

Query

```graphql
query($offset: Int, $limit: Int, $lastPollTime: Time, $sublevelExternalID: String) {
    hierarchyNodeByExternalID(externalId: $sublevelExternalID) {
        ... on SubLevel {
            cases(offset: $offset, limit: $limit, openedAfter: $lastPollTime) {
                ...
            }
        }
    }
}

```

Aggregating Cases to a higher location within the hierarchy...

Assets within Senseye are often too low level for the maintenance system's hierarchy, and the Cases which are raised, need to be aggregated to a higher location within the hierarchy.

This can be achieved by gathering the path to the Case's Asset, along with the External IDs in use. In the example query above, the following section:

```graphql
asset {
    name
    externalId
}

```

can be replaced with the following:

```graphql
asset {
    name
    externalId
    path {
        name
        externalId
    }
}

```

which will provide the name and external id for the asset and all of its parents within the Senseye hierarchy. This can be used to select a suitable depth within the hierarchy for which to raise a Notification within your maintenance system. Please note that Assets can have only one open Case at a time, but when aggregating them up the hierarchy, there may be many open cases at a time on a given hierarchy sub-level.

Senseye can connect directly to your maintenance system to raise notifications in response to a Senseye Case. We support a range of systems out of the box and can develop custom integrations as required.

Your maintenance system must be made accessible to Senseye. In terms of networking, we use a single egress IP address which can be whitelisted within firewalls. If required, we can support VPNs. We recommend that encryption is used during transit and a trusted certificate is applied to the maintenance system. Furthermore, we require suitable authentication credentials and the correct level of access to create notifications.

Please contact [support](mailto:senseye.support.industry@siemens.com) for more information on this topic.

Senseye PdM supports the following single sign on (SSO) protocols:

- Active Directory / LDAP
- OAuth 2.0
- OpenID Connect
- SAML
- WS-Federation

In addition, we have specific support for the following identity providers:

- ADFS
- Google G Suite
- PingFederate
- Azure Active Directory

## Requirements

Given the range of SSO integrations we support, this page does not go into the specifics of each integration. However, in general, Senseye will require the following information:

- The type of SSO integration that is required
- The email domain which will be used with the SSO integration
- Integration specific information; typically a URL to your identity provider's metadata file

To support SSO, there will be some level of configuration required within your identity provider, the following information is typically required:

- Senseye's relaying party is `urn:auth0.senseye` and callback URL is `https://senseye.eu.auth0.com/login/callback`
- Senseye requires name and email address for users
- Ideally the ID used for users would not contain personally identifiable information (e.g. email addresses)
- If you require request signing, please let us know and we can provide a certificate

## Authentication Process

In diagrammatic terms, the following sequence happens when a user using SSO accesses Senseye PdM. Senseye PdM currently uses [Auth0](https://auth0.com) in its authorization toolchain.

To connect your ADFS provider to Senseye, you will need to provide us with either your ADFS URL or a Federation Metadata File for your ADFS.

From your ADFS server you will then need to do the following;

- Open the Microsoft Management Console (mmc.exe).
- Add the AD FS Management Snap-in (from the File > Add/Remove Snap-in menu).
- In the navigation tree select Console Root > AD FS > Relying Party Trusts.
- Select Add Relying Party Trust... from the Actions panel.
- Select Claims Aware and click Start.
- Select Enter data about the relying party manually and click Next.
- Enter a suitable display name for the relying party such as Senseye and click Next.
- Leave the encryption certificate details as default and click Next
- Tick the Enable support for the WS-Federation Passive protocol checkbox and enter `https://senseye.eu.auth0.com/login/callback` for the Relying party WS-Federation Passive protocol URL and click Next.
- Add `urn:auth0:senseye` as an additional relying party trust identifier.
- Specify an access control policy which applies to your organization. This will control which users from your directory are able to use this relying party trust to log in to Senseye.
- Confirm the details and click Finish to add the relying party trust.
- Highlight the newly created relying party trust and select Edit Claim Issuance Policy from the Actions panel.
- Click Add Rule...
- Select Send Claims Using a Custom Rule from the claim rule template dropdown and click Next.
- Give the rule a suitable name such as Senseye Claims and add the following rule

```text
c:[Type == "http://schemas.microsoft.com/ws/2008/06/identity/claims/windowsaccountname", Issuer ==
"AD AUTHORITY"]
=> issue(store = "Active Directory", types =
("http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress",
"http://schemas.xmlsoap.org/ws/2005/05/identity/claims/name",
"http://schemas.xmlsoap.org/ws/2005/05/identity/claims/nameidentifier",
"http://schemas.xmlsoap.org/ws/2005/05/identity/claims/givenname",
"http://schemas.xmlsoap.org/ws/2005/05/identity/claims/surname"), query =
";mail,displayName,objectGUID,givenName,sn;{0}", param = c.Value);

```

Note: This will map `mail`, `displayName`, `objectGUID`, `givenName` and `sn` claims from ADFS to the claims that Auth0 expects.

This rule can be altered according to your directory setup, however, as a bare minimum we require;

- The `http://schemas.xmlsoap.org/ws/2005/05/identity/claims/nameidentifier` claim which should be a unique key for each user and should not contain sensitive information such as an email address.
- The `http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress` claim in order for Senseye notifications to work for users.
- The `http://schemas.xmlsoap.org/ws/2005/05/identity/claims/name` claim.

# Work Orders

Completed Work Orders are represented within Senseye as Work Events. Documenting the work which is performed on an asset is important for the machine learning within Senseye PdM to learn how failures develop and provide context to changes in machine data. The Senseye PdM app allows work to be entered manually, but ideally it would be automatically sourced from your maintenance system.

This section describes the [structure of Senseye Work Events](structure.html) and how to integrate your maintenance system with Senseye, either by [pushing to Senseye's APIs](push.html), or Senseye [connecting directly to your maintenance system](pull/_index.html).

Senseye PdM has an extensive [GraphQL API](../introduction/graphql.html) that can be used to create work events. Alongside [External ID](../introduction/externalids.html) support, it is often selected as a low friction method for Work Order integration.

## Mutation

The following GraphQL query makes use of the [createWorkEvent mutation](https://graphqldocs.senseye.io/#mutation-createWorkEvent) to create a work event within Senseye:

Mutation

```graphql
mutation createWorkEvent($input: CreateWorkEventInput!) {
    createWorkEvent(input: $input) {
        workEvent {
            id
            creationTime
        }
    }
}

```

Variables

```json
{
    "input": {
        "eventTime": "2022-04-03T20:00:00+01:00",
        "message": "Replaced seized bearing on the drive shaft",
        "externalID": "site-A|asset-id",
        "isRoutine": false,
        "workActivity": "repair",
        "failureMode": "MaPS:FAILURE_MODE"
    }
}

```

More information about the input variables can be found within [work event structure](structure.html) documentation.

If you want to use Senseye's ID...

The following is the same example, but using Senseye's ID instead of the `externalID`:

Variables

```json
{
    "input": {
        "eventTime": "2022-04-03T20:00:00+01:00",
        "message": "Replaced seized bearing on the drive shaft",
        "hierarchyNodeID": "senseye-asset-guid",
        "isRoutine": false,
        "workActivity": "repair",
        "failureMode": "MaPS:FAILURE_MODE"
    }
}

```

Work events within Senseye are composed of the following required attributes:

- **Timestamp**: the date, time and time zone that the work took place
- **Message**: A free text description of the work undertaken
- **Asset ID**: Either the Senseye or External ID of the asset the work was performed on
- **Is Routine**: Whether the event was routine or non-routine
- **Work Activity**: The type of work undertaken

In addition to these required attributes, the following optional attribute can be provided:

- **Failure Mode**: Identifies the failure encountered on the asset

Each will be described in more detail in the following sections.

## Timestamp

The `eventTime` attribute should identify when the work was completed. This allows Senseye to learn how the failure evolved to the point of intervention and implies that any changes in data may be related to the recent work on the asset.

Ideally the date and time provided should be as close to the work being completed as possible. If an accurate timestamp is not available, the application prefers timestamps which are known to be after the work being completed, rather than before.

The timestamp should include date, time and time zone, see the [timestamp](../introduction/timeanddates.html) documentation for more information on the required format.

## Message

The message is a free text description which explains the work. It is shown within the application and often used as contextual information when exploring cases and root causes.

## Asset ID

The asset which the work was performed on can be identified by either its Senseye ID or an External ID. Typically the External ID used in this case would be the ID of the asset within your maintenance system - you can read more about configuring and using External IDs [here](../introduction/externalids.html).

## Is Routine

This boolean attribute identifies whether the work was planned or not. It is used in reporting to allow insight to be gained from your maintenance records.

## Work Activity

Work activity explains what work was undertaken. The values are codified within Senseye, they rarely change and are common across all assets. The current list is:

| Name              | ID                 |
| ----------------- | ------------------ |
| Visual Inspection | `visualinspection` |
| Minor Maintenance | `minormaint`       |
| Reconfiguration   | `programChange`    |
| Replacement       | `replacement`      |
| Major Maintenance | `majormaint`       |
| Repair            | `repair`           |

When interacting with the API, the ID of the work activity should be used.

Getting the latest list...

In most cases, we recommend hard coding the above list into your integration logic, but if required, you can gather the latest list from our [GraphQL API](../introduction/graphql.html). The following query uses the [workActivities query on an asset](https://graphqldocs.senseye.io/#definition-Asset) to gather the latest list of work activities:

query

```graphql
{
    assetByExternalID(id: "external-id-for-asset") {
        workActivities {
            id
            name
        }
    }
}

```

If you wish to use the Senseye asset ID, rather than an External ID, replace `assetByExternalID` with `asset`.

## Failure Mode

Failure mode explains what went wrong. It is not required, but by providing this Senseye can learn more about your assets and their failures, resulting in better diagnostics and prognostics. The values are codified within Senseye, they change infrequently and are common across all assets. The current list is:

| Name                   | ID                                          |
| ---------------------- | ------------------------------------------- |
| Abnormal noise         | `MaPS:FAILURE_MODE`         |
| Abnormal wear          | `MaPS:FAILURE_MODE`          |
| Blocked/Plugged/Choked | `MaPS:FAILURE_MODE` |
| Contaminated           | `MaPS:FAILURE_MODE`           |
| Control malfunction    | `MaPS:FAILURE_MODE`    |
| Corroded               | `MaPS:FAILURE_MODE`               |
| Current Incorrect      | `MaPS:FAILURE_MODE`      |
| Damaged                | `MaPS:FAILURE_MODE`                |
| Excessive wear         | `MaPS:FAILURE_MODE`         |
| Fails test/check       | `MaPS:FAILURE_MODE`       |
| Fitted incorrectly     | `MaPS:FAILURE_MODE`     |
| In/output incorrect    | `MaPS:FAILURE_MODE`    |
| Incomplete fill        | `MaPS:FAILURE_MODE`        |
| Leaking                | `MaPS:FAILURE_MODE`                |
| Level incorrect        | `MaPS:FAILURE_MODE`        |
| Loose                  | `MaPS:FAILURE_MODE`                  |
| Misaligned             | `MaPS:FAILURE_MODE`             |
| No fault found         | `MaPS:FAILURE_MODE`         |
| No rotation            | `MaPS:FAILURE_MODE`            |
| Open Circuit           | `MaPS:FAILURE_MODE`           |
| Other                  | `MaPS:FAILURE_MODE`                  |
| Out of adjustment      | `MaPS:FAILURE_MODE`      |
| Out of Balance         | `MaPS:FAILURE_MODE`         |
| Overheating            | `MaPS:FAILURE_MODE`            |
| Overstressed           | `MaPS:FAILURE_MODE`           |
| Pressure incorrect     | `MaPS:FAILURE_MODE`     |
| Replacement (Obsolete) | `MaPS:FAILURE_MODE` |
| Robbed                 | `MaPS:FAILURE_MODE`                 |
| Seized                 | `MaPS:FAILURE_MODE`                 |
| Sensor Malfunction     | `MaPS:FAILURE_MODE`     |
| Short Circuit          | `MaPS:FAILURE_MODE`          |
| Slow to operate        | `MaPS:FAILURE_MODE`        |
| Temperature incorrect  | `MaPS:FAILURE_MODE`  |
| Vibration              | `MaPS:FAILURE_MODE`              |
| Voltage Incorrect      | `MaPS:FAILURE_MODE`      |

Getting the latest list...

In most cases, we recommend hard coding the above list into your integration logic, but if required, you can gather the latest list from our [GraphQL API](../introduction/graphql.html). The following query uses the [failureModes query on an asset](https://graphqldocs.senseye.io/#definition-Asset) to gather the latest list of failure modes:

query

```graphql
{
    assetByExternalID(id: "external-id-for-asset") {
        failureModes {
            id
            name
        }
    }
}

```

If you wish to use the Senseye asset ID, rather than an External ID, replace `assetByExternalID` with `asset`.

Senseye can connect directly to your maintenance system to create [Work Events](../structure.html) within Senseye in response to completed Work Orders. We support a range of systems out of the box and can develop custom integrations as required.

Your maintenance system must be made accessible to Senseye. In terms of networking, we use a single egress IP address which can be whitelisted within firewalls. If required, we can support VPNs. We recommend that encryption is used during transit and a trusted certificate is applied to the maintenance system. Furthermore, we require suitable authentication credentials and the correct level of access to query for completed Work Orders.

Please contact [support](mailto:senseye.support.industry@siemens.com) for more information on this topic.

eMaint is a computerized maintenance management system (CMMS) from Fluke, described [here](https://www.fluke.com/en-gb/products/fluke-software/emaint-cmms)

Senseye has an integration with one customer such that:

1. A Senseye Case Notification (i.e. where an asset is deemed worthy of attention) triggers an eMaint Work Order Request. The Work Order Request can be inspected alongside the Senseye Case and followed by a maintainer up as necessary
1. Completed eMaint Work Orders trigger creation of Senseye Work Events. Senseye polls the eMaint endpoint and creates Work Orders based upon the detail in completed eMaint orders.

This means engineers and maintainers can use Senseye and eMaint together to diagnose problems with assets and record maintenance in a frictionless way as possible.
# SieSmart documentation

# SieSmart API Changelog

This changelog lists all updates to the SieSmart API, in chronological order.

## API Spec: Version 1.2.10 - April 11, 2026 (Current version)

- Updated enum values for document signing status
- Increased text limit for commentToRiskAnalyst on decision request end
- Reference: [b2b-openapi.yaml](reference/b2b-openapi.yaml)

## API Spec: Version 1.2.8 - February 19, 2026

- New endpoint group for managing guarantors (corporate and personal)

  ► /proposals/{proposal-id}/guarantors/..

- New GET endpoint to view the notification URL that has been set

  ► /notifications/endpoint

- Updated asset object, to allow setting of serial number at proposal creation

- Reference: [b2b-openapi-v1.2.8.yaml](reference/schemas/b2b-openapi-v1.2.8.yaml)

## API Spec: Version 1.2.7 - October 29, 2025

- New endpoint for adding customers bank details (Spain only)

  ► /proposals/{proposal-id}/customer/bank-account

## API Spec: Version 1.2.6 - June 18, 2025

- New endpoint for downloading decision document

  ► /proposals/{proposal-id}/decision/decision-document/download - response in pdf format

## API Spec: Version 1.2.5 - May 15, 2025

- New property added to `signatories`: status

  ► This shows the e-sign status per signatory for documents with multiple signatories

- New endpoint for rate cards

  ► /common/rate-cards - shows rate cards in JSON format

## API Spec: Version 1.2.0 - September 13, 2024

- New enum value added to `DecisionStatusEnum`: DECLINED_INTERNAL

  ► This is used in the decision response body

- Asset model (Spain only)

  ► /common/asset-models - allows searching of asset model sub-category. Also new object assetModel available at proposal creation

- Portfolio/Contract endpoints

  ► These endpoints allow searching for and managing in-life contracts

## Non API Spec: API Access Token Expiration Change

- On 24th June 2024 in test and from 1st July 2024 in production, the API access token expiration will change from 24 hours to 12 hours

## API Spec: Version 1.1.1 - April 11, 2024

- New options added to `ProposalStatusEnum`; DEAL_AMENDMENT and CREDIT_APPROVED

  ► This is used in the proposal response body

## API Spec: Version 1.1.0 - Initial Release

# How can we help you?

Please contact your local Siemens Financial Services (SFS) representative for any questions in the first instance.

If you're unsure who that is, reach out to your local team using the contact details below:

| Country        | Email                                                                         | Telephone            |
| -------------- | ----------------------------------------------------------------------------- | -------------------- |
| United Kingdom | [salesUK.sfs@siemens.com](mailto:salesUK.sfs@siemens.com)                     | 0161 446 5056        |
| Norway         | [salessupport.sfs.no@siemens.com](mailto:salesUK.sfs@siemens.com)             | +47 22 63 30 80      |
| Finland        | [myynti.sfs@siemens.com](mailto:myynti.sfs@siemens.com)                       | +358 (0) 10 511 3050 |
| Sweden         | [salessupport.sfs.se@siemens.com](mailto:salessupport.sfs.se@siemens.com)     | +46 90 200 20 20     |
| Denmark        | [structureadmin.sfs.se@siemens.com](mailto:structureadmin.sfs.se@siemens.com) | +46 8730 79 75       |
| Spain          | [info.sfs.es@siemens.com](mailto:info.sfs.es@siemens.com)                     | +34 915 144 581      |
| France         | [partenaire-sfs.fr@siemens.com](mailto:partenaire-sfs.fr@siemens.com)         | +33 01.85.57.07.81   |
| USA            | [vendorSales.us@siemens.com](mailto:vendorSales.us@siemens.com)               | No number            |

For any technical issues, please contact: [apisupport.sfs@siemens.com](mailto:apisupport.sfs@siemens.com)

Our API team will endeavour to respond to you within a maximum of 72 hours.

# Unlocking Success with the SieSmart API

Discover how Siemens Financial Services (SFS) customers are benefiting from a seamless integration with the SieSmart API.

## Lease Group

[Lease Group](https://leasegroup.co.uk/) are a trusted leasing provider with over 25 years of experience in the equipment and technology market.

- **Challenge**: Lease Group are now established finance specialists in multiple industries. Following their initial focus on smartphone leasing for B2B mobile dealers, they sought a lender that could provide instant credit decisions and approvals, rather than an estimate. The SFS API capabilities perfectly matched Lease Group’s requirements.
- **Solution**: SFS’s expertise in technology-based assets made them an ideal financing partner for Lease Group. The integration of SFS’s platform with Lease Group’s systems via API was completed in just ten weeks. This collaboration resulted in an interface that delivers credit decisions in five seconds and enables order tracking from signing to payout.
- **Benefits**: Lease Group’s partnership with SFS has taken their service and platform to a new level, with a near-perfect approval rate. The swift integration allowed them to rapidly meet market demand and Lease Group can take advantage of new opportunities as they arise, by providing rapid financing decisions.

“Partnering with SFS was a decision we thought through carefully. It wasn’t simply an integration with a lender for the sake of it. We wanted to work with a funder with the same mindset and vision as us, and SFS ticked every box."

**Simon Fabb**, Marketing Director at Lease Group

## Bluestone

[Bluestone](https://www.bluestone.app/) is an independent broker and provider of asset finance to UK organisations in both the private and public sectors.

- **Challenge**: As a highly consultative business, Bluestone faced a challenge: losing commodity transactions in the £5,000-£50,000 range to competitors. To gain a competitive edge, they sought a revolutionary end-to-end online portal that would match customers with funders seamlessly.
- **Solution**: As a pioneer in auto-decisioning, SFS partnered with Bluestone to integrate the SieSmart API into Bluestone’s new portal, connecting it to SFS’ automated credit quotation and decision system. The result was an ‘always-on’ facility, enabling swift and automatic decision-making for deals in the £5,000-£50,000 range.
- **Benefits**: The API connection enables Bluestone to close more business, with high acceptance rates among customers using the portal. Auto-decisioning provides financing decisions in seconds. Bluestone’s portal aligns with their strategic growth strategy, allowing scalable expansion without a significant increase in costly human resources.

“As the largest auto acceptance player of any funder in the market, Siemens Financial Services were the ideal collaborator.​ They've got the greatest appetite, and their speed of service is second to none.”

**Mark Hargreaves**, Finance Director at Bluestone.​

## Avoki

[Avoki](https://www.avoki.com) is an innovative Nordic IT and AV equipment supplier operating across Finland, Sweden, and Norway. The company provides sustainable IT solutions tailored to customer needs, including equipment rental with full lifecycle management and environmental accountability.

- **Challenge**: Avoki wanted to boost sales while helping customers make environmentally responsible IT purchasing decisions. Traditional ownership models required large upfront investments and lacked clear responsibility for equipment lifecycle. Manual processes limited business growth.
- **Solution**: Siemens Financial Services partnered with Avoki to deliver flexible leasing options via the SieSmart digital portal. This integration eliminated manual work and paperwork, enabling instant financing decisions. The SieSmart API also streamlined data exchange and supports future integration with Avoki’s Microsoft Dynamics CRM.
- **Benefits**: Avoki customers now rent IT equipment with 100% of devices reused or recycled at end-of-life. The SieSmart portal has digitized sales processes, improved customer experience, and opened new e-commerce opportunities.

"Working with Siemens Financial Services is a true collaboration, not a hat in hand for money. Funding decisions are made quickly, and discussions are smart and knowledgeable."

**Tomi Tihekari** CEO, Avoki Finland.

- [Overview](overview.html)
- [Solutions](solutions.html)
- [Customer Insights](insights.html)
- Developers
  - [Getting Started](get-started/index.html)
  - [Configuration](get-started/config.html)
  - Common scenarios
    - [Create a proposal](common-scenarios/create-proposal.html)
    - [Submit for a decision](common-scenarios/submit-for-decision.html)
    - [Notifications/WebHooks](common-scenarios/notifications.html)
  - API Resources
    - [API URLs](reference/api-urls.html)
    - [Authentication](reference/api-authentication.html)
    - [API Specification](reference/swagger.html)
    - [Changelog](changelog.html)
- [Contact Us](contact.html)
- [Legal](legal/index.html)

# SieSmart

### Create financing\* proposals for your customers straight from your CRM platform

[Contact us](contact.html "Contact us")

###### \*applies to equipment, machinery and technology assets only. Contact us for further information and terms.

# What is the SieSmart API?

SieSmart API provides a secure, seamless link between your own CRM system and SFS online application management tool, SieSmart.\
As vendors and brokers who work with Siemens Financial Services (SFS), it allows you to easily manage calculations and submit financing proposals for credit decision with just a few clicks.\
All interactions are made through **your own existing CRM system,** making submissions quick, easy and with no double-keying or misalignment between systems.

# A value added solution

### Stay on your current CRM system

- No new software for your team to learn
- Reduce duplication and rekeying of data
- Save time, process applications and get instant credit decisions

### A flexible solution

- Tailor the functionality to suit your needs and CRM platform
- Offer your customers a faster, better service
- Be super-responsive and realise revenue more quickly

### No extra costs & peace of mind from SFS

- SieSmart API is free to use for approved, established introducers (vendors/OEMs/brokers) of SFS
- Integration managed by your trusted in-house developers or IT partners

### Ease of access, ease of use

- Enables access at your convenience
- Access a range of SFS financial solutions for your customers in a few clicks
- End-to-end digital and automated solution

### Fully-supported integration

- Dedicated SFS team to support integration by your developers
- Support can be accessed throughout set up, whenever required
- Available test environments

### Simple, safe & secure

- SieSmart API features in-built security – token based and compliant with your digital strategy
- A dedicated developer portal makes accessing technical data and installation/integration reference intuitive and easy
- Full technical documentation and a range of templates are available to use

# Getting Started & Next Steps

To use SieSmart API, **you need to have an established existing trading relationship with Siemens Financial Services (SFS)**

Don't have an existing trading relationship with SFS? Please [Contact us](contact.html) to discuss.

You accept and sign the API specific End-User Licence Agreement (EULA).

Upon reciept, our SFS developer portal link and test credentials are sent to you.

SieSmart API is integrated into your CRM platform by your developers, based on your specific needs.

Integration is supported by SFS with accessible, easy to understand technical documents and our dedicated team.

End to end testing is signed off by you and SFS before live credentials are issued.

You and your team access SieSmart from within your CRM platform whenever you need.

# Currently available in

##### Finland

##### France

##### Norway

##### Spain

##### Sweden

##### United Kingdom

##### United States

# Explore how to use the SieSmart API

Siemens Financial Services (SFS) offers a range of solutions to integrate the SieSmart API with your CRM system. For example, an integration enables you to submit proposals, provide documents, and perform co-terms efficiently.

## Create your own solution

Given the many features of the SFS API, it might be useful to first speak with our dedicated API team to discuss your ideal solution. However, the final decision on how you wish to use the API is down to you as there are many different use cases.

Below are three potential scenarios for the SieSmart API which can be tailored to meet your specific requirements. Please note that the developer commitment may change and will vary based on the specifics of each customer integration.

Simple

Be up and running with a simple solution that can submit and retrieve a credit decision using only two API calls.

- Make proposal calculations
- Create proposals with minimum requirements
- Receive a credit decision
- Two week development commitment and maximum two hours testing

[Contact Us](contact.html)

Moderate

Create a proposal but with extras.

- Co-terms
- Upgrades
- Fees and special payments
- Four week development commitment and maximum four hours testing

[Contact Us](contact.html)

Expert

Access everything you need to do post proposal submission.

- eSignature
- Portfolio
- Data reports
- Six week development commitment and maximum four hours testing

[Contact Us](contact.html)

## SFS Support & Security

- Dedicated API team to support throughout set up and post integration by your developers
- All integrations must be approved by the API team and an SFS sales representative prior to going live
- The SieSmart API features built-in security that is token-based and compliant with industry standards
- A dedicated developer section with full technical documentation and a range of templates available to use

## FAQs

How much does it cost?

The SieSmart API is free to use and the only costs would be associated with your own IT team.

Can I integrate into any CRM system?

Yes, our API is designed to be flexible and can integrate with most CRM systems.

How long does an integration take?

The duration of an integration depends on the complexity of your requirements, but most integrations can be completed within 2-4 weeks.

Can Siemens do the integration work for me?

Our SieSmart API team don't typically take on integration work for customers. However, please contact our API team to discuss.

Can I use your API to build reports?

Yes, our API allows you to build custom reports based on your specific needs.

We’re submitting proposals via the API, how do we get updates when a credit decision changes?

You can configure webhooks to receive real-time updates whenever a credit decision changes.

Where do I get an API key from?

Keys are only issued by the SieSmart API Team after they have discussed your use case with them.

# Create a proposal

Creating a new proposal is the most common task you will likely perform using our API.

Hint

The example objects below show the information you need to create the proposal - the objects in the response will include more information.

## Different workflows

There are two main ways you can achieve this, depending on your workflow:

1. Adding each object to the proposal

It is possible to create the proposal with only one piece of information - the 'officeId'. This will create the proposal in SieSmart and return you a unique proposalId. You can then make subsequent requests to add the customer, asset(s), calculation etc.

This may fit your workflow, but will require more effort to integrate in this manner. For every modifying request you would need a valid [if-Match header](../get-started/index.html#post-request-precondition).

1. Creating the proposal in one request

The more common route would be to build the complete proposal object and submit it in one request.

## Proposal creation

There is a minimum amount of information that is required.

POST /proposals

Hint

A successful response will include the unique proposal ID in the body. Use this proposal ID for all subsequent requests about this proposal.

```json
{
  "officeId": 0,
  "salesContactId": 0,
  "label": "string",
  "specialConditionText": "string",
  "scheduleAssetsDescription": "string",
  "customer": {
    ...
  },
  "assets": [
    ...
  ],
  "calculation": {
    ...
  }
}

```

The request object requires data about the introducer, customer, assets and calculation.

| Name                      | Type    | Required | Description                                                                                                                    |
| ------------------------- | ------- | -------- | ------------------------------------------------------------------------------------------------------------------------------ |
| officeId                  | integer | true     | Assigns the proposal to the correct office                                                                                     |
| salesContactId            | integer | true\*   | Assigns the correct sales person to the proposal (see [Sales contacts](../get-started/config.html#get-your-sales-contact-ids)) |
| label                     | string  | false    | Adds a custom label to the proposal                                                                                            |
| specialConditionText      | string  | false    | Notes field                                                                                                                    |
| scheduleAssetsDescription | string  | false    | Notes field                                                                                                                    |
| customer                  | object  | true     | see [Customer](#customer)                                                                                                      |
| assets                    | object  | true     | see [Assets](#assets)                                                                                                          |
| calcultion                | object  | true     | see [Calculation](#calculation)                                                                                                |

\* Not mandatory in GB

## Customer

Adding a customer to the proposal differs if they are limited or non-limited.

```json
{
  "customer": {
    "registrationNumber": "string",
    "companyType": "LIMITED",
    "sourceCountryCode": "string"
  }
}

```

| Name               | Type   | Required | Description                                             |
| ------------------ | ------ | -------- | ------------------------------------------------------- |
| registrationNumber | string | true     | The official registration number of the limited company |
| companyType        | string | true     | 'LIMITED' or 'NONLIMITED'                               |
| sourceCountryCode  | string | true     | The country code of the customer, eg. GB                |

### Limited customers

If you already know the customers official company registration number, then you only need to add this as the 'registrationNumber', and keep the 'companyType' as 'LIMITED'.

### Non-limited customers

For non-limited customers, you must first use the customer search endpoint (../introducers/customer-search) to find the customers' unique reference number (this differs from their official customer registration number), then you can use this in the customer object as above, but setting the 'companyType' as 'NONLIMITED'.

### Customer search

If you do not know the customer registration number, you are able to perform a search first using information such as the customers name, postcode etc. The response from this request will include a registration number that you can use in the customer object above.

## Assets

Most financial products require at least one asset to be added to the proposal. If the product is a loan then this is not necessary.

Assets are added using their unique code on our system.

The assets you are allowed to use are governed by the trading agreement, and may also differ between different offices. To get the assets that are available to you:

GET /common/iac-codes

```json
{
  "assets": [
    {
      "quantity": 9999,
      "description": "string",
      "valuePerAsset": 10000000,
      "assetIacCode": "string",
      "serviceAgreement": {
        ...
      },
      "location": "string",
      "condition": "NEW",
      "ageInMonth": 1
    }
  ]
}

```

Assets are represented by an array, meaning you can add multiple assets to the proposal. The sum of the asset values must equal the total amount financed that you specify in the calculation object.

| Name             | Type    | Required | Description                                                                       |
| ---------------- | ------- | -------- | --------------------------------------------------------------------------------- |
| quantity         | integer | true     | The quantity of this type of asset                                                |
| description      | string  | true     | Unique description of the asset, eg. 'Photocopier Model XYZ'                      |
| valuePerAsset    | number  | true     | Cost of each asset                                                                |
| assetIacCode     | string  | true     | Unique identifier for the asset (see [assets](../get-started/config.html#assets)) |
| serviceAgreement | object  | false    | Optional object if there is to be a service agreement for the asset               |
| location         | string  | false    | Physical location of the asset                                                    |
| condition        | string  | true     | Condition of the asset: NEW, REFURBISHED, USED, REMANUFACTURED, PREREGISTERED     |
| ageInMonth       | integer | false\*  | The age of the asset (\*is required if condition is not NEW)                      |

### Service agreement

It is possible to add a service agreement to an individual asset.

```json
{
  "serviceAgreement": {
    "serviceType": "LIMITED",
    "pricePerUnitValues": [
      {
        "pricePerUnit": {
          "id": 0,
          "name": "string"
        },
        "value": 0,
        "valueIncluded": 0
      }
    ]
  }
}

```

| Name               | Type         | Required | Description                              |
| ------------------ | ------------ | -------- | ---------------------------------------- |
| serviceType        | string       | true     | Type of service agreement: LIMITED, FULL |
| pricePerUnitValues | object array | true     | Price information of each unit           |

## Calculation

The calculation object is used for adding details about the amount financed, term, payment method etc.

```json
{
"calculation": {
    "operation": "CALC_RENTAL",
    "sheet": {
      "marketProductId": 0,
      "introducerPricelistId": 0,
      "paymentPeriod": "MONTHLY",
      "paymentMethod": {
        "id": 0,
        "name": "string"
      },
      "paymentMode": "IN_ADVANCE",
      "customerPto": 0,
      "residualValue": {
        "amount": 0,
        "percentage": 0,
        "usePercentage": true,
        "manualSet": true
      },
      "term": 0,
      "commissions": [
        ...
      ],
      "fees": [
        ...
      ],
      "specialPayments": [
        ...
      ],
      "optionFee": 0
    }

```

| Name            | Type         | Required | Description                                                                                |
| --------------- | ------------ | -------- | ------------------------------------------------------------------------------------------ |
| operation       | string       | false    | Calculation to perform. Eg. if amount financed and term are provided, calculate the rental |
| commissions     | object array | false    | Details any commissions paid                                                               |
| fees            | object array | false    | Details any fees paid                                                                      |
| specialPayments | object array | false    | Details any special payments                                                               |

### Commissions

A list of commission types is available from the calculation config endpoint (see [calculation config](../reference/swagger.html)). If this array is empty, there are no commission types available to you.

```json
{
  "commissions": [
    {
      "type": {
        "id": 0,
        "name": "string",
        "readonly": true
      },
      "amount": 0,
      "percentage": 0,
      "usePercentage": true,
      "manualSet": true
    }
  ]
}

```

### Fees

A list of fee types is available from the calculation config endpoint (see [calculation config](../reference/swagger.html)). If this array is empty, there are no fee types available to you.

```json
{
  "fees": [
    {
      "feeType": {
        "id": 0,
        "name": "string",
        "readonly": true,
        "code": "string"
      },
      "name": "string",
      "amount": 0,
      "manualSet": true,
      "feeIncluded": true
    }
  ]
}

```

### Special Payments

A list of special payment types is available from the calculation config endpoint (see [calculation config](../reference/swagger.html)). If this array is empty, there are no special payment types available to you.

```json
{
  "specialPayments": [
    {
      "type": {
        "id": 0,
        "name": "string",
        "readonly": true,
        "downPaymentType": true,
        "code": "string"
      },
      "amount": 0,
      "equipmentDetails": "string",
      "serialNumber": "string"
    }
  ]
}

```

## Example payload

Here is an example of a minimum payload required to create a proposal.

```json
{
  "officeId": 1234,
  "salesContactId": 1234,
  "customer": {
    "registrationNumber": "string",
    "companyType": "LIMITED",
    "sourceCountryCode": "string"
  },
  "assets": [
    {
      "quantity": 1,
      "description": "ACME Photocopier",
      "valuePerAsset": 5000,
      "assetIacCode": "1234",
      "condition": "NEW"
    }
  ],
  "calculation": {
    "operation": "CALC_RENTAL",
    "sheet": {
      "marketProductId": 0,
      "introducerPricelistId": 0,
      "paymentPeriod": "MONTHLY",
      "paymentMethod": {
        "id": 0,
        "name": "string"
      },
      "paymentMode": "IN_ADVANCE",
      "term": 12
    }
  }
}

```

# Notifications/Webhooks

It is possible to subscribe to notifications to receive status updates about the proposals you have submitted. This is a more efficient way of getting updates than polling for changes on a schedule.

Notifications will be sent by Siemens Financial Services (SFS) in certain scenarios:

- When the credit decision changes; if the deal isn't initially auto-accepted, the credit decision is subject to change when it has been assessed by our under-writers (DECISION_RECEIVED)
- When the overall status of the proposal changes; for example when the contract has been activated (CREDIT_STATUS_CHANGED)

Hint

You will need the ability to receive incoming API requests to make use of the notification service.

## Setting up initial notification service

To be able to receive notifications from SFS, you first need to register your notification endpoint. There are three endpoints under the heading 'NotificationEndpoints' that allow you to do this.

This is a single URL which will receive notifications about all proposals you subscribe for:

POST /notifications/endpoint

```json
{
  "apiKey": "sdJH67ZZHHJj23dsf",
  "basePath": "https://myb2bhost.com/b2b/api/siemart"
}

```

| Name     | Type   | Required | Description                                                                                                                     |
| -------- | ------ | -------- | ------------------------------------------------------------------------------------------------------------------------------- |
| apiKey   | string | false    | A unique key that will be sent back to you in the request header, as X-API-KEY                                                  |
| basePath | string | true     | The URL where POST requests from SFS will be sent. You should register your base path; we will add '/notifications' to the URL. |

Hint

- You only need to perform this step once.
- The URL you register is the base URL. We will add '/notifications' to this when sending the notification. For example if you register `https://myb2bhost.com/b2b/api/siemart`, we will send notifications to `https://myb2bhost.com/b2b/api/siemart/notifications`.

There are also PUT and DELETE requests that allow you to modify the above information, or cancel notifications overall.

## Subscribing to proposal notifications

Once you have completed the step above, you need to subscribe to notifications for each proposal you are interested in.

A separate POST request needs to be made to subscribe to each event type.

Example:

POST /notifications?proposal-id=1234&event-type=DECISION_RECEIVED

The response to this will be a UUID - this is a unique ID identifying that combination of proposal ID and event type.

Important

Save this UUID in your system, as when a notification is sent to you, you will need the UUID to identify it.

## Handling notifications

When a notification that you have subscribed to is triggered, a POST request will be sent to the URL you registered. The UUID will be included in the 'notification-uuid' query parameter.

For example:

```text
https://myb2bhost.com/b2b/api/siesmart/notifications?notification-uuid=ee3f630f-89d1-4c4b-8ed0-60e3cb7a1ab5

```

The body of the POST request will be empty.

If you registered an API Key as part of the initial registration, this will be included here as an X-API-KEY header.

The request that is sent to you doesn't contain any information about the proposal, or the event that triggered it, just the UUID.

You should then query the proposals endpoint to get the latest status.

# Submit for a decision

Once you have created the proposal you can submit it for a credit decision. For this to happen all the pre-conditions have to be fulfilled.

## Decision pre-check (optional)

This allows you to check that all pre-conditions have been fulfilled and that you are able to submit for a credit decision.

GET /proposals/{proposal-id}/decision/pre-check

If allowed, you will see 'requestAllowed: true'. If not the response will highlight what is wrong.

```json
{
  "requestAllowed": true,
  "cancelAllowed": true,
  "preconditionViolations": [
    {
      "code": "string",
      "userMessage": "string"
    }
  ]
}

```

If the requestAllowed is false, please check your proposal creation set up, or correct the violations and try again.

## Submit for decision

If you have had a positive result to the pre-check response (optional) above, or are confident that what you have created is correct, you may submit it for a credit decision.

POST /proposals/{proposal-id}/decision/request

```json
{
  "commentToRiskAnalyst": "string",
  "siemensMayContactCustomer": false,
  "siemensMayRequestCreditData": false,
  "prelease": false
}

```

| Name                          | Type    | Required | Description                                          |
| ----------------------------- | ------- | -------- | ---------------------------------------------------- |
| commentToRiskAnalyst          | string  | false    | Optional comment to the underwriter                  |
| siemensMayContactCustomer\*   | boolean | false    | Details if Siemens needs to contact customer         |
| siemensMayRequestCreditData\* | boolean | false    | Credit data request - US only proposals              |
| prelease\*                    | boolean | false    | Payment required in advance of equipment being built |

\*mandatory fields

A successful response will include the entire proposal object, including a decision object:

```json
{
  "decision": {
    "decisionStatus": "NOT_AVAILABLE",
    "decisionRequestedDate": "2023-06-15T10:14:03.968Z",
    "decisionDate": "2023-06-15T10:14:03.968Z",
    "decisionValidUntilDate": "2023-06-15T10:14:03.968Z",
    "comment": "string",
    "condition": "string",
    "reason": "string",
    "commentToRiskAnalyst": "string",
    "siemensMayContactCustomer": true
  }
}

```

If you're missing the ETag, once fetched, add it into the If-Match paramater seen here in a Postman example:

### Decision Statuses

The credit decision status can be one of the following:

| Label                     | Description                                                  |
| ------------------------- | ------------------------------------------------------------ |
| NOT_AVAILABLE             | There is no decision status available                        |
| PENDING                   | The proposal has been sent to manual underwriting for review |
| ACCEPTED                  | The proposal has been credit approved                        |
| DECLINED                  | The proposal is declined                                     |
| MORE_INFORMATION_REQUIRED | More information is needed to progress                       |
| CANCELLED                 | The proposal has been cancelled                              |
| UNKNOWN                   | Issue with proposal, will need a manual review               |
| AMEND                     | The proposal is credit approved but deal amendment required  |
| ACCEPTED_WITH_CONDITIONS  | The proposal has been accepted but with conditions           |
| EXPIRED                   | The proposal has expired                                     |

Hint

The credit decision status is not the same as the proposal status.

# Overview

Getting started with our API is straight-forward using the guide below.

## Terminology

| Term         | Meaning                                                                                                                                                                                                                                            |
| ------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Access token | A token is retrieved by the Client after a successful access authorization flow. The access token is used by the API to authenticate Clients. The access token does not contain any information that the Client (or client's application) can use. |
| SieSmart     | The backend system to which the API connects.                                                                                                                                                                                                      |
| Introducer   | Collective name for you as the user, representing either a broker or vendor.                                                                                                                                                                       |

## Prerequisites

- A trading agreement with Siemens Financial Services (SFS) - [contact us](../contact.html) to get this
- A signed End-User License Agreement (EULA) that covers our API connection
- Credentials (client ID and secret)

## Differences between production and test APIs

The production version of the API provides access to real data. The system (SieSmart) is also connected to live applications such as credit search engines. The test environment is where you can develop your applications before you are ready to promote them to production. Config items, for example pricelist IDs, are consistent across the test and production environments.

The test environment is subject to regular database refreshes, therefore any data you have sent is liable to be periodically deleted.

## Promoting your connection to production

Production API credentials are only provided by SFS after we are satisfied that your application is working correctly in the test environment. This will include (but not limited to) verifying that proposals you have sent have all the correct information. There is no limit to the amount of tests you are allowed to perform.

When everything is in order, the production credentials will be generated and sent to you.

## Security considerations

Security is an important aspect. This API is developed by using well-tested and widely adopted security schemes. The Access Token are sent in the header of the request. All the communication between the API and client is secured by TLS, which encrypts the traffic. When developing an application, it is important to keep the credentials and tokens safe and handle them carefully. If you believe your credentials have been compromised please [contact SFS](../contact.html) immediately.

## HTTP Response codes

SFS uses conventional HTTP response codes to indicate the success or failure of an API request. In general: Codes in the 2xx range indicate success. Codes in the 4xx range indicate an error that failed given the information provided (e.g., a required parameter was omitted, the token has expired, etc.). Codes in the 5xx range indicate an error with our servers.

General responses to API requests

| Status | Meaning               | Description                                              | Schema               |
| ------ | --------------------- | -------------------------------------------------------- | -------------------- |
| 200    | OK                    | The request has been successful                          | Dependent on request |
| 401    | Unauthorized          | The auth token is invalid or not present                 | Error                |
| 403    | Forbidden             | The requested operation is not authorized for your token | Error                |
| 412    | Precondition failed   | For POST requests, the 'If-Match' header is missing      | Error                |
| 429    | Too Many Requests     | The request rate limit has been reached                  | Error                |
| 500    | Internal server error | Internal server error                                    | Error                |
| 503    | Service unavailable   | Service unavailable                                      | Error                |

## POST request precondition

For any modifying calls (POST, PUT, PATCH, DELETE) there is a mandatory If-Match request header. The purpose of this is to make sure you are modifying the most recent version of the proposal. This header is a string that represents a date time object.

The value for this can be retrieved from the ETag response header from the initial proposal creation, or by making a GET request to the proposal endpoint.

Here is an example of where you can capture your ETag when testing in Postman:

Hint

- The If-Match request header is not needed for the initial proposal creation call

## Request IDs

Each API request has an associated request identifier. You can find this value in the response headers, under `X-B2B-TRACE-ID`. You can also find request identifiers in the Error object of unsuccessful requests. If you provide this request identifier when contacting SFS it will allow us to debug and help you as quickly as possible.

## Rate limiting

There is a limit of 100 requests per minute, enforced at the connecting IP address level. If you reach this limit you will receive an HTTP 429 Too Many Requests response status code. If your request rate falls below this limit then the block will be automatically removed.

## Connecting to the API

To be able to connect to our API, there are a few requirements:

- Client ID
- Client Secret
- Access token ([generate a token](../reference/api-authentication.html))

If you do not have your Client ID and Client Secret, please [contact us](../contact.html).

The generated token is valid for 12 hours and must be sent in the header of every request.

Once you are connected successfully, there are a number of config endpoints that will help you get started.

[Configuration](config.html)

# Configuration

## Introducer

### Get your Office ID(s)

Depending on your set up in SieSmart you may have one or more offices. 'Offices' don't just refer to physical locations, they may be used to assign different configurations for different industries, for example.

GET /common/offices

```json
[
  {
    "id": 1,
    "name": "ACME Industries - Commercial Brokers",
    "postalAddress": {
      ...
    }
  },
  {
    "id": 2,
    "name": "ACME Industries - Machine Tools",
    "postalAddress": {
      ...
    }
  }
]

```

### Get your Sales contact IDs

When creating a proposal, you have to set the sales person who is creating the deal\*. Omitting this will set the sales person to the generic API User.

Sales people could vary across offices (if you have more than one office), so a required parameter is the office ID.

\* Mandatory in all countries except UK

GET /common/sales-persons

```json
[
  {
    "id": 23076571,
    "firstName": "API",
    "lastName": "TechUser",
    "title": {
      "id": 45,
      "name": "Ms"
    },
    "email": "api-user@siemens.com",
    "phone": null,
    "cellPhone": "",
    "fax": null,
    "role": {
      "alias": "ROC_SAM",
      "name": "Sales Manager"
    },
    "blocked": false,
    "blockedReason": null,
    "loginPresent": true,
    "officeIds": [1234]
  }
]

```

## Assets

To create a proposal for most financial products, at least one asset is necessary. These are added to the proposal using their unique IAC code. The list of IAC codes that are available to you are dependent on the office, as different offices may deal with different sectors

GET /common/iac-codes?office-id=

```json
[
  {
    "iacCode": "ITE2201",
    "name": "Desktop standalone PC"
  },
  {
    "iacCode": "ITE",
    "name": "Computer Equipment"
  }
]

```

## Calculation

To create a calculation (and a proposal), there are several configurations that you will need to use. In general, these are the available market products, pricelists, payment methods, terms etc.

These configurations can vary across offices, so it is important you check the config endpoint to make sure you can successfully create a calculation.

GET /calculation/config

```json
{
  "config": {
    "officeId": 0,
    "office": {
      ...
    },
    "marketProducts": [
      ...
    ],
    "pricelists": [
      {
        "id": 0,
        "name": "string"
      }
    ],
    "paymentPeriods": [
      "MONTHLY",
      "BIMONTHLY",
      "QUARTER",
      "TRIANNUALLY",
      "HALFANNUALLY",
      "ANNUALLY",
      null
    ],
    "paymentMethods": [
      {
        "id": 0,
        "name": "string"
      }
    ],
    "currency": {
      "currencyCode": "string",
      "format": "string",
      "fractionDigits": 0
    },
    "interestRateConfig": {
      "min": 0,
      "max": 0,
      "fractionDigits": 0
    },
    "tariffConfig": {
      "fractionDigits": 0,
      "fractionDigitsNordics": 0
    },
    "assetValueConfig": {
      "min": 0,
      "max": 0
    },
    "residualValueConfig": {
      "percentageMin": 0,
      "percentageMax": 0,
      "readonly": true
    },
    "termConfig": {
      "min": 0,
      "max": 0
    },
    "postponedStartAffectedParties": [
      {
        "id": 0,
        "name": "string"
      }
    ],
    "commissionTypes": [
      {
        "id": 0,
        "name": "string",
        "readonly": true,
        "editable": true
      }
    ],
    "feeTypes": [
      {
        "id": 0,
        "name": "string",
        "readonly": true,
        "editable": true,
        "code": "string"
      }
    ],
    "specialPaymentTypes": [
      {
        "id": 0,
        "name": "string",
        "readonly": true,
        "editable": true,
        "downPaymentType": true,
        "code": "string"
      }
    ],
    "allowedOperations": [
      "CALC_RENTAL"
    ],
    "autoCalculateVat": true
  },
  "calculationSheet": {
    ...
  }
}

```

| Name           | Type         | Description                                     |
| -------------- | ------------ | ----------------------------------------------- |
| officeId       | integer      | The office that this configuration is valid for |
| office         | object       | Details about the office                        |
| marketProducts | object array | see [Market products](#market-products)         |
| priceLists     | object array | see [Price lists](#price-lists)                 |
| paymentPeriods | object array | see [Payment periods](#payment-periods)         |

### Market products

We offer different market products for financing, for example Leasing (fixed and minimum term), hire purchase. This array will show what products you are able to use.

```json
{
  "marketProducts": [
    {
      "id": 0,
      "name": "string",
      "loanOrHirePurchaseInNordics": true,
      "hpAnnuityOrStraightAmortization": true,
      "hirePurchaseInNordics": true
    }
  ]
}

```

| Name                            | Type    | Description                                               |
| ------------------------------- | ------- | --------------------------------------------------------- |
| id                              | integer | Unique ID of the product. Necessary for proposal creation |
| name                            | string  | Name of the product (eg. HP)                              |
| loanOrHirePurchaseInNordics     | boolean | Nordics only - `true` or `false`                          |
| hpAnnuityOrStraightAmortization | boolean | Nordics only - `true` or `false`                          |
| hirePurchaseInNordics           | boolean | Nordics only - `true` or `false`                          |

### Price lists

Typically there is one price list for an office, but there could be more than one, for instance for different market products. The price list affects the calculation.

### Payment periods

The available payment periods.

# End User License Agreement (EULA)

End User License Agreement for Intermediaries and Vendors

Terms and Conditions of Use

1 About our Terms

1.1 These Terms explain how you may use SieSmart, My SFS and the SieSmart API (the SFS Technology), each of which simplifies the lease administration process and helps you to take full advantage of our services.

1.2 You should read these Terms carefully before using the SFS Technology. By accessing or using the SFS Technology or otherwise indicating your consent, you acknowledge that you have understood and agree to be bound by these Terms and the documents referred to in them. If at any time you do not agree to these Terms, you must immediately terminate all use of the SFS Technology. If you have any questions about the SFS Technology, please contact us through your SFS representative.

1.3 Definitions

Confidential Information means any and all information made available to you through the SFS Technology which is not already generally available to the public.

Customer means the counterparty to any asset finance agreement.

IP Rights means rights such as copyright, trade marks, domain names, design rights, database rights, patents and all other intellectual property rights of any kind whether or not they are registered or unregistered (anywhere in the world).

License has the meaning given to it in clause 4.1.

My SFS means our Progressive Web Application which allows you to access our services from mobile and desktop devices.

OWASP Top Ten means the standard awareness document for developers and web application security published by the Open Web Application Security Project and available at `https://owasp.org/www-project-top-ten/`.

SieSmart means the web platform which allows you to access our services online, as further described in clause 2 below.

SieSmart API means our application programming interface, which is to be used as an interface between SieSmart and your applications.

SieSmart API Documentation means the documents (in whatever media) that accompany the SieSmart API including our policies for use of the SieSmart API.

SFS Content means all data, text, images, video, audio or other multimedia content, software or other information or material which is uploaded to or processed via the SFS Technology other than the User Content, including but not limited to any materials and data provided by us and accessed by you, such as online forms, tables and these Terms.

Terms means these terms and conditions of use as updated from time to time under clause 8.

User Content means any text, images, video, audio or other multimedia content, software or other information or material which you process through, submit or upload to or on the SFS Technology.

We, us or our means Siemens Financial Services Limited (company registration number 00646166) and the registered office of which is at Sefton Park, Bells Hill, Stoke Poges, Buckinghamshire, SL2 4JS. References to "us" in these Terms also includes our group companies from time to time.

You or your means the person accessing or using the SFS Technology or the SFS Content.

2 Using the SFS Technology

2.1 The SFS Technology is for your commercial use only and is designed to accelerate the lease administration process through the use of SieSmart to generate lease quotations. It is not intended to operate as a replacement for any of the more detailed terms and processes set out in any trading agreement or similar arrangement between us.

2.2 You agree that you will only use the SFS Technology in accordance with these Terms and such other instructions from us concerning your use of the SFS Technology as may be amended or revised by us at our discretion at any time.

2.3 You agree that you will only use the SieSmart API in accordance with the SieSmart API Documentation and that you will notify us as soon as you suspect or become aware of any use of the SieSmart API otherwise than in accordance with the SieSmart API Documentation. In using the SieSmart API you agree that you will act in accordance with best practice including but not limited to the OWASP Top Ten.

2.4 You will not attempt to circumvent any limitations placed by us on use of the SieSmart API without our consent. Such limitations are imposed by us at our sole discretion.

2.5 You agree that you are solely responsible for:

(a) all costs and expenses you may incur in relation to your use of the SFS Technology;\
(b) keeping your user name, password and other credentials confidential and ensuring that such information is not stolen, mislaid or used in an unauthorized manner\
© any activities and actions that take place through your account by use of your credentials (and that you will be liable for all actions and activities that take place through your account by use of your credentials) whether authorized or not;\
(d) your own hardware, the User Content and any other data uploaded through or to the SFS Technology; and\
(e) third parties unlawfully obtaining access to your accounts in order to abuse the nature and intent of the SFS Technology.

2.6 You further agree that:

(a) you will only use the SFS Technology for the purpose of introducing Customers to us and/or our group companies and not for any other purpose unless you obtain our prior written consent;\
(b) you will keep all Confidential Information strictly confidential, and that you will not disclose such information to any other person;\
© you will inform us immediately if you have any reason to believe that there has been any unauthorized use of your user name, password and/or other account details.; and\
(d) you are satisfied that the SFS Technology is suitable for the purpose for which you propose to use it.

2.7 You will ensure that your use of the SFS Technology is in accordance with all relevant legislation, regulations, codes of practice, guidance and other requirements of any relevant government, governmental or regulatory agency or other relevant body.

2.8 You will not use the SFS Technology:

(a) in a manner that will cause interruption, damage, impairment or reduction in the quality or depth of the SFS Technology, servers or networks providing the SFS Technology or the services that we offer to you, Customers or other suppliers through the SFS Technology;\
(b) in connection with a criminal offense under the applicable national laws or regulations or against public order or applicable ethical standards and codes;\
© in any way which causes or is intended to cause annoyance, inconvenience or needless anxiety;\
(d) for any unlawful purpose whatsoever, including fraud or terrorism;\
(e) in any way which is abusive, harmful, threatening or defamatory or any other way that may cause offense;\
(f) in any way that could be harmful to our systems or data (including uploading any material that otherwise contains a virus or other malicious code);\
(g) in any way which breaches or could potentially breach a legal duty to a third party (including a duty of confidentiality) or which infringes or could potentially infringe a person's right to privacy;\
(h) in any way which promotes discrimination or is likely to incite hatred; or\
(i) in any way which may infringe the intellectual property rights of third parties or which promotes any unlawful act.

2.9 You will not:

(a) distribute, license, sell, rent, lease or otherwise deal in or encumber the SFS Technology;\
(b) modify, add to, or otherwise enhance the SFS Technology;\
© build databases or create permanent copies of data scraped from the SFS Technology;\
(d) reverse engineer or extract the source code from the SFS Technology; or\
(e) copy the SFS Technology.

2.10 Your use of the SFS Technology is intended only for access from within the UK. If you choose to access the SFS Technology from locations outside the UK, you are responsible for compliance with local laws where they are applicable.

2.11 We reserve the right to:

(a) terminate your License and your access to the SFS Technology immediately if:

(i) you do not properly perform your obligations under these Terms;\
(ii) you fail to comply with the instructions for use of the services from time to time in force as displayed on the SFS Technology or in the SieSmart API Documentation; or\
(iii) any representations or warranties you have made to us are or become incorrect,

(b) monitor your use of the SFS Technology in order to ensure quality, improve our products and services and to verify compliance with these Terms;\
© make modifications to the SFS Technology at any time for any reason; and\
(d) terminate or withdraw any of the SFS Technology at any time without prior notice and for any reason.

2.12 On termination or discontinuation of access to the SFS Technology you will immediately delete any stored SFS Content.

3 Data Protection and Use of Information

For the purposes of this clause 3, the following definitions shall apply:

Controller has the meaning given in applicable Data Protection Laws from time to time;

Data Protection Laws means, as binding on either party or the service provided under these Terms:

a) the GDPR;\
b) the Data Protection Act 2018;\
c) any laws which implement any such laws; and\
d) any laws that replace, extend, re-enact, consolidate or amend any of the foregoing;

Data Subject has the meaning given in applicable Data Protection Laws from time to time;

GDPR means the General Data Protection Regulation, Regulation (EU) 2016/679;

Personal Data has the meaning given in applicable Data Protection Laws from time to time;

Processing has the meaning given in applicable Data Protection Laws from time to time (and related expressions, including process, processed, processing, and processes shall be construed accordingly);

Processor has the meaning given in applicable Data Protection Laws from time to time; and

Protected Data means Personal Data received from or on behalf of you in connection with the performance of our obligations under these Terms.

3.1 We understand that our use of Personal Data requires trust. We are committed to the highest standards of data privacy and will only use Protected Data for clearly described purposes and in accordance with Data Subjects’ data protection rights. We will comply with our obligations under Data Protection Laws in respect of Protected Data. We may disclose Protected Data for the purposes of checking credit, arranging insurance, administering the customer agreement and preventing fraud, to credit reference agencies, insurers and other finance companies. We will not use Protected Data for marketing or other purposes or disclose that information to other companies for such purposes, without first giving the Customer the opportunity of opting out. For further details about our use of Personal Data please read our Privacy Notice and Fair Processing Notice.

3.2 For the purposes of these Terms you and we are deemed to be separate Controllers, in respect of the Processing of Protected Data in connection with this Agreement. You and we shall comply with Data Protection Laws in Processing Protected Data. You will not by your act or omission cause us to breach Data Protection Laws. You will ensure that you have obtained appropriate consents (or otherwise have a lawful basis under GDPR) for providing the Protected Data to us. Furthermore, you shall provide appropriate fair processing information (including our fair processing notices as updated from time to time) to Data Subjects identified in the Protected Data.

3.3 On the basis of your business relationship with us, we will continue to keep you informed by email about our financial products which you may wish to recommend to your clients. If you would like to opt out of marketing communications, you can click the 'Unsubscribe' link at the bottom of any marketing email we send to you.

4 Ownership and Intellectual Property Rights

4.1 We grant you a limited, non-exclusive, non-sublicensable, fully paid, non-transferable, royalty-free license to use the SFS Technology and the SFS Content within the UK and only for the purposes authorized in clause 2 of these Terms (License). Using the SFS Technology in an unlawful way is expressly prohibited under these Terms and may result in civil and criminal penalties. Your License will continue indefinitely until such time as it is terminated in accordance with clause 2.11 above.

4.2 Except for the License, all IP Rights in and to the SFS Technology and the SFS Content shall vest and remain vested in us. To the extent that you acquire any IP Rights in the SFS Technology or the SFS Content, you will assign or procure the assignment of such IP Rights with full title guarantee (including by way of present assignment of future IP Rights) to us or to anyone else we nominate. You will execute all such documents and do such things as we may consider necessary to give effect to this clause.

4.3 You grant to us a royalty-free, perpetual, irrevocable, non-exclusive, transferable license to use, reproduce, modify, publish, edit, translate, distribute, perform and display any User Content and any data or analytics generated from the same.

4.4 SieSmart and MySFS are trade marks of Siemens Financial Services GmbH. Other trade marks, logos and trade names may also be used within the SFS Technology. The use of any trade marks, logos and trade names is strictly prohibited unless you have our prior written permission.

5 Links to Third Party Websites

5.1 We may provide links to sites on the Internet via the SFS Technology which are operated by other people. If you use these links, you will be leaving the SFS Technology and we therefore take no responsibility in respect of linked sites.

6 Warranties

6.1 You warrant that:

(a) you have the right, power and authority to agree to these Terms and to grant to us the rights contemplated in these Terms;\
(b) you hold all rights and have obtained all licenses and consents required to use the SFS Technology; and\
© your use of the SFS Technology will not infringe any rights of any third party or our rights, nor will it breach any applicable laws or regulations, including Data Protection Laws.

6.2 The SFS Technology is provided on an ‘AS IS’ basis and you acknowledge and agree that the SFS Technology may not be free of bugs or errors and agrees that the existence of minor bugs or errors shall not constitute a breach of these Terms. We do not warrant that the SFS Technology will be uninterrupted or error-free or compatible with third party software or equipment.

6.3 Neither we nor our suppliers give any warranties nor make any representations about the SFS Content or about results to be obtained from using the SFS Technology and shall not be liable for any loss or damage arising out of any virus or other malicious code.

6.4 Any warranties we provide under these Terms will be subject to you using the SFS Technology in compliance with these Terms and the SieSmart API Documentation, and we will not be liable under these Terms for, or required to remedy, any problem arising from:

(a) any modification made to any part of the SFS Technology by anyone other than us without our express prior written consent; or\
(b) any defect or error wholly caused by any equipment or third-party software used in connection with the SFS Technology.

6.5 Subject to these express Terms and to the maximum extent permitted by applicable law, we hereby disclaim all other warranties, terms and conditions, either express, implied or statutory in relation to the SFS Technology.

6.6 You should notify us as soon as possible of any fault or problem you are experiencing with the SFS Technology by contacting your SFS representative who will be able to conduct the lease administration process for your lease business if the SFS Technology is not available. Any errors will be corrected at our sole discretion. We may, but are not obliged to, use any feedback or suggestions you provide including by way of publicity.

7 Rights of third parties

No one other than a party to these Terms has any right to enforce any of these Terms.

8 Variation

These Terms are dated June 2020. No changes to these Terms are valid or have any effect unless agreed by us in writing. We reserve the right to vary these Terms from time to time. Our updated terms will be displayed on SieSmart and My SFS and communicated to you directly if needed and by continuing to use and access the SFS Technology following such changes, you agree to be bound by any variation made by us. It is your responsibility to check these Terms to verify such variations.

# Authentication

Authentication is handled using OAuth 2.0 Client Credentials Grant.

**Token URLs:**

| Environment       | Endpoint URL                                             |
| ----------------- | -------------------------------------------------------- |
| Test:             | `https://sfs-q-001.siemens-sfs.auth0app.com/oauth/token` |
| Live (Production) | `https://sfs-p-001.siemens-sfs.auth0app.com/oauth/token` |

**API Audiences:**

| Environment | Audience                       |
| ----------- | ------------------------------ |
| Test        | `https://api.siesmart.com/b2b` |
| Production  | `https://api.siesmart.com/b2b` |

## Retrieve an access token

To be able to make any API calls, you will need to generate an access token:

```json
POST {TOKEN_URL}

Content-Type: application/json

{
  "audience": "{API_AUDIENCE}",
  "grant_type": "client_credentials",
  "client_id": "{YOUR_CLIENT_ID}",
  "client_secret": "{YOUR_CLIENT_SECRET}"
}

```

A successful authorization request results in a response like the following:

```json
HTTP/1.1 200 OK

Content-Type: application/json

{
  "access_token": "eyJz93a...k4laUWw",
  "token_type": "Bearer",
  "expires_in": 86400
}

```

The field `access_token` contains the token which you use for performing requests to our APIs.

Hint

Tokens are valid for 12 hours so can be cached on your system. Please avoid a set-up that generates a new token for each request.

# API URLs

Our APIs are available through different base URLs. To access these APIs, combine the **base URL + version + endpoint**, for example to "create a proposal" construct the URL to call like this: `Base URL for environment` + `/version number` + `/proposals/proposal`

eg: `https://api-test.sfs.siemens.cloud/siesmart/b2b/v1/proposals/proposal`

Use the testing environment to get familiarized with API behaviour and try out things. And use the live environment to start real transactions with Siemens Financial Services (SFS).

Important

Use HTTPS for all of your interactions with SFS APIs.

## Versioning

The URL is subject to change if a new version is released.

Current version: v1

## Base URL - Testing (playground)

- `https://api-test.sfs.siemens.cloud/siesmart/b2b/`

## Base URL - Live (production)

- `https://api.sfs.siemens.cloud/siesmart/b2b/`

# API Specification
# SiGREEN documentation

# Concepts and Glossary

| **Term**                     | **Description**                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           | **Example**                                                                                                                                                                                                                                                           |
| ---------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| System Generated IDs (UUID)  | Universally Unique Identifier (UUID) is a 36-character alphanumeric string that can be used to identify information. A standard UUID code contains 32 hex digits along with 4 “-” symbols, which makes its length equal to 36 characters. To enhance the flexibility, interoperability, and consistent unique identification on the SiGREEN platform, system-defined UUIDs (Universally Unique Identifiers) are implemented to uniquely identify objects on the platform. These UUIDs provide a reliable and standardized method to uniquely identify individual objects within SiGREEN. Whenever a new component, product, supplier, or any other entity is created within SiGREEN platform, the system automatically generates and assigns a UUID to that object. This ensures that every object can be uniquely identified throughout the application without any manual intervention. | C99BE31E-3698-422D-91E3-8270102CC7D5                                                                                                                                                                                                                                  |
| Component Identifier Type    | The term ‘component identifier type’ is assigned by each company for the components. This is used to group and categorize different identifiers.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          | - Stock Keeping Units code (SKU) - Chemical Abstracts Service (CAS) - SAP Number - Component/Part Numbers (CPID) - Universal Product code (UPC) - Batch or Lot Numbers - Manufacturing Codes                                                                          |
| Component Identifier         | A component identifier is a unique code or ID assigned by each company to identify individual component. It is used to track and group components within a company’s internal systems. A component identifier is typically an alpha-numerical code.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       | Component Identifiers for the Component Identifier Type “CAS number” can look like: - 85-01-8 - 206-06-3 - 15-87-9                                                                                                                                                    |
| Product Identifier Type      | The term 'product identifier type’ is assigned by each company for the products. This is used to group and categorize different identifiers.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              | - Universal Product code (UPC) - European Article Number (EAN) - Global Trade Item Number (GTIN) - Manufacturer Part Number (MPN)                                                                                                                                     |
| Product Identifier           | A product identifier is a unique code or ID assigned by each company to identify individual products. A product identifier can be a unique code or label assigned to a specific product. It serves to distinguish one product from another within a company's inventory. A product identifier is typically an alpha-numerical code.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       | Product identifiers for the product identifier type “Universal Product Code (UPC)” can look like: - 68348784779 - 85849794836 - 58786089502                                                                                                                           |
| Supplier Identifier Type     | The term 'supplier identifier type’ is assigned by each company for the suppliers. This is used to group and categorize different identifiers.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            | - Supplier ID or Code - Business Partner Number - Data Universal Numbering System (DUNS) - Vendor IDs - Tax ID or VAT Numbers - Global Location Numbers (GLN) - Certification Codes - Location ID                                                                     |
| Supplier Identifier          | A supplier identifier is a unique code or label assigned by each company to identify and track their suppliers individually. It is used to manage the relationship with the suppliers and ensure efficient communication with external partners. A supplier identifier is typically an alpha-numerical code.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              | Supplier identifiers for the supplier identifier type “Data Universal Numbering System (DUNS)” can look like: - 10045674 - 10045673 - 10075867                                                                                                                        |
| Factories                    | A factory refers to a site or manufacturing facility where the last manufacturing or assembly step to produce a product Factories takes place. They are typically represented as locations or factory codes.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              | - City: Munich, Germany - Site code: MUC-001 - Factory name: River Rouge                                                                                                                                                                                              |
| Component                    | Physical goods, Products, materials, software order services purchased from an upstream supplier; used as input for the manufacturing of products. Synonyms for a component can include ‘material’, ‘purchased good’, or ‘article.’                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |                                                                                                                                                                                                                                                                       |
| Component PCF                | Component PCF (Product Carbon Footprint) refers to a product carbon footprint documentation of a Product, material, or service, produced by a supplier. If not directly produced by a supplier, it can also be an assembly of components from suppliers used as a module in a product.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    | My PCF is a Component PCF for my customer.                                                                                                                                                                                                                            |
| Component PCF (aggregated)   | Component PCF (aggregated) returns only aggregated values such as component, totalPcf, and pcfUnit.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       | A PCF documentation opposed to a disaggregated PCF, which contains the information of which components or materials and processes a product is made of and what their individual contributions are to the overall PCF (matched Bill of Material and Bill of Process). |
| Component PCF (limited data) | Component PCF (limited data) returns more values than the aggregated values, which includes dataQuality, typeOfQuality, Transparency, and percentage.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     | -                                                                                                                                                                                                                                                                     |
| dataQuality                  | When receiving or documenting a PCF in SiGREEN, the application provides a quality indicator section. The metrics in this section provides indications on the level of confidence that can be derived from the PCF received or PCF created.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               | -                                                                                                                                                                                                                                                                     |
| totalPcf                     | The total PCF is the sum of all sub-categories of a pcf but of only one life cycle stage. There are potentially two total PCF values for each life cycle stage.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           | For the life cycle stage production: GWP total (excl. bio. CO2) = +GWP fossil +GWP biogenic CO2e-emissions (only other GHG emissions than CO2 – excludes biogenic CO2) +GWP direct land use change (dLUC) +GWP land use (LU) +GWP aircraft emissions                  |
| pcfUnit                      | A product carbon footprint is measured in units of carbon dioxide for example kg CO2 equivalent (CO2e) per unit of product or tonnes CO2e. The selection of an appropriate unit is important as it provides a unit of measurement to the consumer that reflects the quantity of product that is used by the end user.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     | -                                                                                                                                                                                                                                                                     |
| Transparency                 | Transparency or Documentation is the quality of the documentation provided by the user when disclosing the PCF information. Measured as the unweighted share of attributes provided out of the maximum number of attributes which can be used for documentation.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          | -                                                                                                                                                                                                                                                                     |
| Reliability                  | Reliability or Assurance describes the level of audit from self-declaration. It's an internal PCF review which provides a score of 25% and third party verification provides a score of 100%.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             | -                                                                                                                                                                                                                                                                     |
| Validity                     | Validity & Data Accuracy or Data Quality Rating (DQR) measure technological, temporal, and geographical representativeness and in some cases historically additionally completeness and reliability.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      | -                                                                                                                                                                                                                                                                     |
| Supplier                     | A supplier refers to an entity or organization that provides goods, materials, or services to a company for use in its products or operations. Suppliers play a crucial role in the supply chain and can significantly impact the environmental footprint of products through their production processes, transportation methods, and sourcing practices.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 | - F&B Supplier - Metal One - Plastic & Metal - Peter Landscape                                                                                                                                                                                                        |
| location                     | A supplier refers to an entity or organization that provides goods, materials, or services to a company for use in its products or operations. Suppliers play a crucial role in the supply chain and can significantly impact the environmental footprint of products through their production processes, transportation methods, and sourcing practices.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 | - Berlin - Munich - EMEA - France                                                                                                                                                                                                                                     |
| Product                      | Any good (tangible product, such as material) or service (intangible product) that a company produces and sells. It is a tangible item or good that is manufactured, distributed, and consumed. It encompasses a wide range of goods across various industries, each with its unique lifecycle and associated environmental impacts. Products can include everything from consumer goods like electronics and clothing to industrial equipment and machinery.                                                                                                                                                                                                                                                                                                                                                                                                                             | - eRod - Protein Bar - Household Appliances - Smartphone                                                                                                                                                                                                              |
| family                       | A product family is a group of related products that share similar characteristics and are designed to meet specific customer needs.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      | - Rear Axle - Front Axle                                                                                                                                                                                                                                              |
| identifiers                  | Identifiers are unique codes or labels assigned to entities such as products, components, or suppliers for easy tracking and differentiation. Each identifier includes information about its type, value, and whether it serves as the primary identifier for the entity.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 | -                                                                                                                                                                                                                                                                     |
| idType                       | A product ID type is a standardized or customized identifier assigned to a product for tracking and management purposes. It can be a recognized standard like the Universal Product Code (UPC), or a customized naming system created by the user.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        | - Universal Product Code (UPC) - Identifier 2                                                                                                                                                                                                                         |
| value                        | A value of product ID type is a unique code or identifier assigned to a specific product, allowing for easy tracking and identification within a database or system. It typically consists of alphanumeric characters.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    | 02035711268                                                                                                                                                                                                                                                           |
| default                      | Default refers to a setting or value that is chosen or applied. In the context of identifiers, default can be true or false. When set to true, it indicates that the identifier is the main or primary identifier for the entity. Conversely, when set to false, it means that the identifier is not the main identifier.Default refers to a setting or value that is chosen or applied. In the context of identifiers, default can be true or false. When set to true, it indicates that the identifier is the main or primary identifier for the entity. Conversely, when set to false, it means that the identifier is not the main identifier.                                                                                                                                                                                                                                        | - true - false                                                                                                                                                                                                                                                        |

# Contact

For any inquiries, whether organizational matters like the procurement of API credentials or technical questions about the Sigreen API, please direct your communication to the following email: [sigreensupport.industry@siemens.com](mailto:sigreensupport.industry@siemens.com)

# Getting Started with our Application

- Please find our Getting Started Guide of the SiGREEN application here: [Getting Started with the application](https://siemens-help.stonly.com/kb/en)

## Getting Started with our APIs

To begin using the **SiGREEN API**, follow these steps:

1. Request SiGREEN API access from the SiGREEN team via [sigreensupport.industry@siemens.com](mailto:sigreensupport.industry@siemens.com)
1. After our approval process you will get Client ID and a Client Secret.
1. Make a request against our identity provider SiemensID to receive a JWT by using Client ID and Client Secret provided by SiGREEN team.
1. Include the JWT Token `Authorization` header of your HTTP requests.

## Request a Token

| Environment  | Siemens ID URL                                    |
| ------------ | ------------------------------------------------- |
| SiGREEN UAT  | `https://siemens-00340.eu.auth0.com/oauth/token/` |
| SiGREEN PROD | `https://siemens-00236.eu.auth0.com/oauth/token/` |

```bash
curl https://siemens-00236.eu.auth0.com/oauth/token \ <-- Please reference to the table above regarding different environments

  -H 'content-type: application/json' \
  -d "{
            \"client_id\":\"$CLIENT_ID\",
            \"client_secret\":\"$CLIENT_SECRET\",
            \"audience\":\"https://app.sigreen.siemens.com/\",
            \"grant_type\":\"client_credentials\"
      }" 

```

## Example response

```bash
{
  "access_token": "jwtHeader.jwtPayload.jwtSignature",
  "token_type": "Bearer",
  "expires_in": 1800
}

```

## Example request to SiGREEN APIs

| Environment  | SiGREEN Base URL                                       |
| ------------ | ------------------------------------------------------ |
| SiGREEN UAT  | `https://app-uat.sigreen-playground.siemens.cloud/api` |
| SiGREEN PROD | `https://app.sigreen.siemens.com/api`                  |

```bash
curl --request POST \
  --url https://app.sigreen.siemens.com/api/products/factoryEmissions/42564 \
  --header 'Authorization: Bearer jwtHeader.jwtPayload.jwtSignature'

```

- [Overview](overview.html)
- [Getting Started](getting-started.html)
- SiGREEN APIs
  - Procurement API(s)
    - [Quick Start](procurement-api/quickstart-procurement.html)
    - [API Reference](procurement-api/api-procurement.html)
    - [Changelog](procurement-api/changelog.html)
  - Product API(s)
    - [Quick Start](product-api/quickstart-product.html)
    - [API Reference](product-api/api-product.html)
    - [Changelog](product-api/changelog.html)
  - My Tasks API(s)
    - [Quick Start](my-tasks-api/quickstart-my-tasks.html)
    - [API Reference](my-tasks-api/api-my-tasks.html)
    - [Changelog](my-tasks-api/changelog.html)
  - PCF APIs
    - [Quick Start](pcf-api/quickstart-pcf-api.html)
    - [API Reference](pcf-api/api-pcf.html)
    - [Changelog](pcf-api/changelog.html)
- [Validation Errors](validation-error.html)
- [Concepts and Glossary](concepts-glossary.html)
- [Contact](contact.html)

# Overview

## SiGREEN API

Application Programming Interfaces (APIs) play a critical role in today's interconnected digital landscape. They serve as a bridge between disparate software systems, allowing them to seamlessly communicate and exchange data. APIs are essential for streamlining processes, enhancing collaboration, and driving innovation across industries. By embracing APIs, businesses can unlock new opportunities, improve operational efficiency, and stay ahead in a rapidly evolving marketplace.

## Unleash the potential of SiGREEN APIs

SiGREEN offers a suite of powerful APIs designed to revolutionize your PCF management practices. With SiGREEN API, you can easily connect your Enterprise Resource Planning (ERP) and Product Lifecycle Management (PLM) systems to SiGREEN.

Seamlessly retrieve critical product, supplier and material data and integrate it into your existing workflows. This streamlined data exchange ensures smooth and efficient product decarbonization efforts.

Unlock the full potential of our SiGREEN API, and experience the convenience of an all-in-one solution for managing product lifecycles and securing your business's future success.

## Master Data Management

Efficiently synchronize your material, supplier and product master data with SiGREEN.

## Product Carbon Footprint

Export detailed carbon footprints of your products. Use this information for further analysis, enrich your product data in PLM systems or catalogs, and make data-driven decisions.

## Supply Chain Emissions

Enhance your materials data by incorporating actual product carbon footprints from your supply chain partners. SiGREEN APIs allow you to export emissions data from sourced components and materials, providing comprehensive insight into the carbon footprint of your supply chain.

## Concepts and Glossary

For detailed information about the concepts and terminology, please refer to [Concepts and Glossary](concepts-glossary.html).

## Overview of APIs

### Procurement APIs

#### Supplier APIs

| **Task** | **Description**                                |
| -------- | ---------------------------------------------- |
| GET      | Get supplier by ID for one company in SiGREEN. |
| PUT      | Update a supplier by supplier ID in SiGREEN.   |
| DELETE   | Delete supplier by supplier ID in SiGREEN.     |
| GET      | Get all suppliers for a company in SiGREEN.    |
| POST     | Create a supplier in SiGREEN.                  |

#### Request PCF APIs

| **Task** | **Description**                           |
| -------- | ----------------------------------------- |
| POST     | Send PCF request to suppliers in SiGREEN. |

#### Component Secondary Data APIs

| **Task** | **Description**                                            |
| -------- | ---------------------------------------------------------- |
| GET      | Get component secondary data by secondary data ID.         |
| PUT      | Update component secondary data by secondary data ID.      |
| GET      | Get all component secondary data for component in SiGREEN. |
| POST     | Create secondary data with component.                      |

#### Supplier Identifier Type APIs

| **Task** | **Description**                                                     |
| -------- | ------------------------------------------------------------------- |
| GET      | Get supplier identifier types for a company in SiGREEN.             |
| POST     | Create a supplier identifier type for a company in SiGREEN.         |
| DELETE   | Delete an unused supplier identifier type for a company in SiGREEN. |
| PATCH    | Update a supplier identifier type for a company in SiGREEN.         |

#### Component Identifier Types APIs

| **Task** | **Description**                                                      |
| -------- | -------------------------------------------------------------------- |
| GET      | Get component identifier types for a company in SiGREEN.             |
| POST     | Create a component identifier type for a company in SiGREEN.         |
| DELETE   | Delete an unused component identifier type for a company in SiGREEN. |
| PATCH    | Update a component identifier type for a company in SiGREEN.         |

#### Get Component's PCF APIs

| **Task** | **Description**                                                       |
| -------- | --------------------------------------------------------------------- |
| GET      | Get an export of a PCF of a component without cryptographic material. |
| GET      | Get the PCF data for your components in SiGREEN.                      |
| GET      | Get an component average emission.                                    |

#### Suppliers Invitation APIs

| **Task** | **Description**                       |
| -------- | ------------------------------------- |
| POST     | Invite supplier to SiGREEN via email. |
| POST     | Cancel supplier invite to SiGREEN.    |

#### Component Supplier APIs

| **Task** | **Description**                                                                    |
| -------- | ---------------------------------------------------------------------------------- |
| GET      | Get linked suppliers by componentSupplierLink ID with component.                   |
| PUT      | Update location/share/secondary data/supplier product ID by component supplier ID. |
| DELETE   | Delete link between component and supplier.                                        |
| GET      | Get all linked suppliers with component.                                           |
| POST     | Link suppliers and secondary data with component.                                  |

#### Components APIs

| **Task** | **Description**                                |
| -------- | ---------------------------------------------- |
| GET      | Get component details by ID.                   |
| PUT      | Update component details by ID.                |
| DELETE   | Delete component by ID.                        |
| GET      | Get all components for one company in SiGREEN. |
| POST     | Create component in SiGREEN.                   |

#### Material Categories APIs

| **Task** | **Description**                                         |
| -------- | ------------------------------------------------------- |
| GET      | Get Material Category details by ID.                    |
| PUT      | Update Material Categories details by ID.               |
| DELETE   | Delete Material Categories details by ID.               |
| GET      | Get Material Category image by ID.                      |
| PUT      | Update Material Category image by ID.                   |
| DELETE   | Delete Material Category image by ID.                   |
| GET      | Get all Material Categories for one company in SiGREEN. |
| POST     | Create Material Category to SiGREEN.                    |

### Product APIs

#### Product APIs

| **Task** | **Description**                           |
| -------- | ----------------------------------------- |
| GET      | Get product details by ID.                |
| PUT      | Update a product in SiGREEN.              |
| DELETE   | Delete a product in SiGREEN.              |
| GET      | Get all products of a company in SiGREEN. |
| POST     | Upload a product to SiGREEN.              |

#### Bill of Materials Version APIs

| **Task** | **Description**                                            |
| -------- | ---------------------------------------------------------- |
| POST     | Create Bill of Material versions for a product in SiGREEN. |
| DELETE   | Delete a BOM version of a product in SiGREEN.              |
| PATCH    | Update the BOM version of a product in SiGREEN.            |
| GET      | Get a list of BOM versions.                                |

#### Product Identifier Type APIs

| **Task** | **Description**                                                    |
| -------- | ------------------------------------------------------------------ |
| GET      | Get product identifier types for a company in SiGREEN.             |
| POST     | Create a product identifier type for a company in SiGREEN.         |
| DELETE   | Delete an unused product identifier type for a company in SiGREEN. |
| PATCH    | Update a product identifier type for a company in SiGREEN.         |

#### Product PCF APIs

| **Task** | **Description**                                |
| -------- | ---------------------------------------------- |
| GET      | Get product PCF details by ID.                 |
| GET      | Get the PCF data for your products in SiGREEN. |

#### Factory APIs

| **Task** | **Description**                                    |
| -------- | -------------------------------------------------- |
| GET      | Get factories for a company in SiGREEN.            |
| POST     | Create a factory for a company in SiGREEN.         |
| DELETE   | Delete an unused factory for a company in SiGREEN. |
| PATCH    | Update a factory for a company in SiGREEN.         |

#### Bill of Material APIs

| **Task** | **Description**                                                                                                                                                                                                                                                         |
| -------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| GET      | Get a list of Bill of Material entries for a product in SiGREEN.                                                                                                                                                                                                        |
| PATCH    | Add Groups, Components, and Products to product BOM version in SiGREEN, and return the job Id. Job Status API can be used to retrieve the status of operations. This API performs tasks such as adding BOM items, synchronizing components, and executing calculations. |
| PATCH    | Save the current state of the upstream emission data for a product BOM version in SiGREEN. Once saved, the upstream emissions BOM will be locked, and no further changes will be allowed.                                                                               |
| DELETE   | Delete BOM item (Groups, Components and Products) to product BOM version in SiGREEN.                                                                                                                                                                                    |
| PATCH    | Update meta data of Groups, Components and Products to product BOM version in SiGREEN.                                                                                                                                                                                  |

#### Job APIs

| **Task** | **Description**                                |
| -------- | ---------------------------------------------- |
| GET      | This API is used to fetch the status of a job. |

#### Factory Emissions APIs

| **Task** | **Description**                                                                                                                         |
| -------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| POST     | Upload factory emissions for a product in SiGREEN. Please note that this API endpoint is marked as deprecated and will be removed soon. |

#### Factory Emissions APIs

| **Task** | **Description**                                    |
| -------- | -------------------------------------------------- |
| POST     | Upload factory emissions for a product in SiGREEN. |
| DELETE   | Delete factory emissions for a product in SiGREEN. |

### My Tasks APIs

#### Product Tasks APIs

| **Task** | **Description**                              |
| -------- | -------------------------------------------- |
| GET      | GET the incoming PCF requests of my company. |
| PUT      | Link product to the task.                    |
| DELETE   | Delete link between product and task.        |
| PUT      | Share the PCF data with the given task.      |

#### Component Tasks APIs

| **Task** | **Description**                               |
| -------- | --------------------------------------------- |
| GET      | GET the outbound PCF requests of my company.  |
| PATCH    | Accept proof received to the task in SIGREEN. |
| PATCH    | Reject proof received to the task in SIGREEN. |

#### Task Comments APIs

| **Task** | **Description**                                |
| -------- | ---------------------------------------------- |
| GET      | Get comments for the given task in SIGREEN.    |
| POST     | Add a comment to the given task in SIGREEN.    |
| PUT      | Update a comment to the given task in SIGREEN. |
| DELETE   | Delete a comment to the given task in SIGREEN. |

### PCF APIs

#### Product’s PCF APIs

| **Task** | **Description**                                                                         |
| -------- | --------------------------------------------------------------------------------------- |
| POST     | To create a TFS standard credential for a specific product of a company in SiGREEN.     |
| POST     | To create a PACT standard credential for a specific product of a company in SiGREEN.    |
| POST     | To create an ISO standard credential for a specific product of a company in SiGREEN.    |
| POST     | To create a CatenaX standard credential for a specific product of a company in SiGREEN. |
| GET      | It is to know whether the credential has been created or not.                           |

## Contact

For general information about the application, the APIs, any issues or feature requests please contact:

Email: [sigreensupport.industry@siemens.com](mailto:sigreensupport.industry@siemens.com)

# Validation Errors

## Validation Errors

| **Validation Error Code** | **Error Description**                                                                                                                            |
| ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ |
| SIG-VLD-0001              | An error occurred while parsing the token. The token provided is invalid.                                                                        |
| SIG-VLD-0002              | Invalid request payload.                                                                                                                         |
| SIG-VLD-0003              | The size requested exceeds the maximum allowable page size.                                                                                      |
| SIG-VLD-0004              | The client Id should not be null or empty                                                                                                        |
| SIG-VLD-0005              | The product upload data list must have at least one item.                                                                                        |
| SIG-VLD-0006              | Product identifier (`%s`) value cannot be more than 100 characters.                                                                              |
| SIG-VLD-0007              | Duplicate product id (UUID).                                                                                                                     |
| SIG-VLD-0008              | Duplicate main product identifier.                                                                                                               |
| SIG-VLD-0009              | You cannot have duplicate identifiers in the identifier list.                                                                                    |
| SIG-VLD-0010              | Duplicate identifier value not allowed in the identifier list.                                                                                   |
| SIG-VLD-0011              | The identifier list should not contain more than one main identifier (default).                                                                  |
| SIG-VLD-0012              | The identifier list must include at least one main identifier(default).                                                                          |
| SIG-VLD-0013              | Invalid identifier value `%s`, it cannot contain \<,>,",”,= characters.                                                                          |
| SIG-VLD-0014              | Product name cannot be blank.                                                                                                                    |
| SIG-VLD-0015              | Product name cannot be more than 100 characters.                                                                                                 |
| SIG-VLD-0016              | Product name cannot contain \<,>,",”,= characters.                                                                                               |
| SIG-VLD-0017              | Main identifier is a mandatory attribute.                                                                                                        |
| SIG-VLD-0018              | Product description cannot be more than 500 characters.                                                                                          |
| SIG-VLD-0019              | Weight cannot be more than 15 digits.                                                                                                            |
| SIG-VLD-0020              | Quantity cannot be more than 15 digits.                                                                                                          |
| SIG-VLD-0021              | Product Identifier type cannot be more than 100 characters.                                                                                      |
| SIG-VLD-0022              | Product Identifier type cannot be empty.                                                                                                         |
| SIG-VLD-0023              | Product identifier value cannot be more than 100 characters.                                                                                     |
| SIG-VLD-0024              | Product identifier value cannot be empty.                                                                                                        |
| SIG-VLD-0025              | Description cannot contain \<,>,",”,= characters.                                                                                                |
| SIG-VLD-0026              | Invalid quantity.                                                                                                                                |
| SIG-VLD-0027              | Invalid weight.                                                                                                                                  |
| SIG-VLD-0028              | Invalid product id.                                                                                                                              |
| SIG-VLD-0029              | Invalid or unknown parameter present in request.                                                                                                 |
| SIG-VLD-0030              | Requested page number value is invalid.                                                                                                          |
| SIG-VLD-0031              | Requested page size value is invalid.                                                                                                            |
| SIG-VLD-0032              | Requested page size exceeded maximum allowed page size.                                                                                          |
| SIG-VLD-0033              | Invalid range in date fields, check from and to date range.                                                                                      |
| SIG-VLD-0034              | Future dates are not permitted in request.                                                                                                       |
| SIG-VLD-0035              | Product family name cannot be more than 100 characters.                                                                                          |
| SIG-VLD-0036              | Family name cannot contain \<,>,",”,= characters.                                                                                                |
| SIG-VLD-0037              | Unknown column names present in request.                                                                                                         |
| SIG-VLD-0038              | Invalid credential filter provided.                                                                                                              |
| SIG-VLD-0039              | Identifier type cannot contain \<,>,",”,= characters.                                                                                            |
| SIG-VLD-0040              | Factory name value cannot be empty.                                                                                                              |
| SIG-VLD-0041              | Factory name cannot be more than 100 characters.                                                                                                 |
| SIG-VLD-0042              | Factory name cannot contain \<,>,",”,= characters.                                                                                               |
| SIG-VLD-0043              | Invalid Factory id.                                                                                                                              |
| SIG-VLD-0044              | Request cannot be empty.                                                                                                                         |
| SIG-VLD-0045              | Duplicate Main Component Identifier.                                                                                                             |
| SIG-VLD-0046              | Component name cannot contain \<,>,",”,= characters.                                                                                             |
| SIG-VLD-0047              | Material category cannot contain \<,>,",”,= characters.                                                                                          |
| SIG-VLD-0048              | Unit type must not be null or empty.                                                                                                             |
| SIG-VLD-0049              | Identifier count cannot exceed the maximum limit of 100.                                                                                         |
| SIG-VLD-0050              | Identifier type must not be null or empty.                                                                                                       |
| SIG-VLD-0051              | Identifier value must not be null or empty.                                                                                                      |
| SIG-VLD-0052              | Component name must not be null or empty.                                                                                                        |
| SIG-VLD-0053              | Component name must be less than or equal to 100 characters.                                                                                     |
| SIG-VLD-0054              | Identifier value must be less than or equal to 100 characters.                                                                                   |
| SIG-VLD-0055              | Supplier Product Id must be less than or equal to 100 characters.                                                                                |
| SIG-VLD-0056              | Identifier type must be less than or equal to 100 characters.                                                                                    |
| SIG-VLD-0057              | Component Id (UUID) must be less than or equal to 36 characters.                                                                                 |
| SIG-VLD-0058              | Invalid Component Id (UUID).                                                                                                                     |
| SIG-VLD-0059              | Supplier Id (UUID) must be less than or equal to 36 characters.                                                                                  |
| SIG-VLD-0060              | Invalid Supplier Id (UUID).                                                                                                                      |
| SIG-VLD-0061              | Share must be less than or equal to 15 characters.                                                                                               |
| SIG-VLD-0062              | Share is invalid.                                                                                                                                |
| SIG-VLD-0063              | Material Category must be less than or equal to 100 characters.                                                                                  |
| SIG-VLD-0064              | Supplier name must not be null or empty.                                                                                                         |
| SIG-VLD-0065              | Supplier name must be less than or equal to 100 characters.                                                                                      |
| SIG-VLD-0066              | Email must not be null or empty.                                                                                                                 |
| SIG-VLD-0067              | Supplier's Email must be less than or equal to 50 characters.                                                                                    |
| SIG-VLD-0068              | Email is invalid.                                                                                                                                |
| SIG-VLD-0069              | Website is invalid.                                                                                                                              |
| SIG-VLD-0070              | Contact must be valid.                                                                                                                           |
| SIG-VLD-0071              | Location must not be null or empty.                                                                                                              |
| SIG-VLD-0072              | Duplicate supplier location.                                                                                                                     |
| SIG-VLD-0073              | Maximum `%s` locations allowed.                                                                                                                  |
| SIG-VLD-0074              | Secondary data id must be less than or equal to 36 characters.                                                                                   |
| SIG-VLD-0075              | Invalid Secondary Data Id (UUID).                                                                                                                |
| SIG-VLD-0076              | Supplier Id (UUID) must not be null or empty                                                                                                     |
| SIG-VLD-0077              | Supplier contact number is limited to 30 character                                                                                               |
| SIG-VLD-0078              | Comment cannot be more than 1000 characters.                                                                                                     |
| SIG-VLD-0079              | Supplier's location must be less than or equal to 200 characters.                                                                                |
| SIG-VLD-0080              | Supplier's website must be less than or equal to 200 characters.                                                                                 |
| SIG-VLD-0081              | Invalid Component Supplier Id (UUID).                                                                                                            |
| SIG-VLD-0082              | Supplier locations `%s` cannot be found, which are linked to components                                                                          |
| SIG-VLD-0083              | Invalid Date Format. Please send valid date format (for eg: "")                                                                                  |
| SIG-VLD-0084              | To date is missing.                                                                                                                              |
| SIG-VLD-0085              | Invalid Date.                                                                                                                                    |
| SIG-VLD-0086              | From date is missing.                                                                                                                            |
| SIG-VLD-0087              | Empty values are not allowed in locations array. Please provide valid locations.                                                                 |
| SIG-VLD-0088              | Component Id (UUID) must not be null or empty                                                                                                    |
| SIG-VLD-0089              | Invalid Total incl. Bio for `%s`                                                                                                                 |
| SIG-VLD-0090              | Invalid Total excl. Bio for `%s`                                                                                                                 |
| SIG-VLD-0091              | Invalid Fossil for `%s`                                                                                                                          |
| SIG-VLD-0092              | Invalid Biogenic for `%s`                                                                                                                        |
| SIG-VLD-0093              | Invalid dLuc for `%s`                                                                                                                            |
| SIG-VLD-0094              | Invalid Land Use for `%s`                                                                                                                        |
| SIG-VLD-0095              | Invalid Aircraft for `%s`                                                                                                                        |
| SIG-VLD-0096              | Quantity must be greater than zero.                                                                                                              |
| SIG-VLD-0097              | Quantity must not be null or empty.                                                                                                              |
| SIG-VLD-0098              | TotalInclBio and TotalExclBio are empty for production stage, one of them is mandatory.                                                          |
| SIG-VLD-0099              | Invalid Data Verification                                                                                                                        |
| SIG-VLD-0100              | Request param length cannot exceed 1000 characters.                                                                                              |
| SIG-VLD-0101              | ComponentIds list must not be empty.                                                                                                             |
| SIG-VLD-0102              | pcfId value must be less than or equal to 100 characters.                                                                                        |
| SIG-VLD-0103              | Duplicate Main Supplier Identifier                                                                                                               |
| SIG-VLD-0104              | Email cannot be more than 50 characters.                                                                                                         |
| SIG-VLD-0105              | Data Source cannot be more than 100 characters.                                                                                                  |
| SIG-VLD-0106              | Search text cannot be more than 50 characters.                                                                                                   |
| SIG-VLD-0107              | Data Verification cannot be more than 50 characters.                                                                                             |
| SIG-VLD-0108              | Total incl. Bio cannot be more than 15 digits.                                                                                                   |
| SIG-VLD-0109              | Total excl. Bio cannot be more than 15 digits.                                                                                                   |
| SIG-VLD-0110              | Fossil cannot be more than 15 digits.                                                                                                            |
| SIG-VLD-0111              | Aircraft cannot be more than 15 digits.                                                                                                          |
| SIG-VLD-0112              | Land Use cannot be more than 15 digits.                                                                                                          |
| SIG-VLD-0113              | dLuc cannot be more than 15 digits.                                                                                                              |
| SIG-VLD-0114              | Biogenic cannot be more than 15 digits.                                                                                                          |
| SIG-VLD-0115              | Region cannot be more than 200 characters.                                                                                                       |
| SIG-VLD-0116              | validityPeriodStart has invalid Date Format. Please send valid date format (for eg: "")                                                          |
| SIG-VLD-0117              | Component Supplier Id (UUID) must not be null or empty                                                                                           |
| SIG-VLD-0118              | Component Supplier Id (UUID) must be less than or equal to 36 characters.                                                                        |
| SIG-VLD-0119              | Secondary Data Id (UUID) must not be null or empty                                                                                               |
| SIG-VLD-0120              | Secondary Data Id (UUID) must be less than or equal to 36 characters.                                                                            |
| SIG-VLD-0121              | Invalid supplier identifier type id                                                                                                              |
| SIG-VLD-0122              | Supplier identifier type id must not be null or empty.                                                                                           |
| SIG-VLD-0123              | Requested page value is invalid. Please send a valid page value.                                                                                 |
| SIG-VLD-0124              | Requested size value is invalid. Please send a valid size value.                                                                                 |
| SIG-VLD-0125              | Requested size value exceeded the limit.                                                                                                         |
| SIG-VLD-0126              | Unexpected `%s`.                                                                                                                                 |
| SIG-VLD-0127              | From date is greater than To date                                                                                                                |
| SIG-VLD-0128              | User not found for user ID `%s` and company ID `%s`.                                                                                             |
| SIG-VLD-0129              | Company not found for company ID `%s`.                                                                                                           |
| SIG-VLD-0130              | User don't have the `%s` role.                                                                                                                   |
| SIG-VLD-0131              | Task status is not New. Only New task can be `%s`.                                                                                               |
| SIG-VLD-0132              | Task not found for this task UID `%s`.                                                                                                           |
| SIG-VLD-0133              | Task not a `%s` task.                                                                                                                            |
| SIG-VLD-0134              | From date or to date values must not be empty.                                                                                                   |
| SIG-VLD-0135              | Invalid sort parameter entered.                                                                                                                  |
| SIG-VLD-0136              | Invalid status parameter entered.                                                                                                                |
| SIG-VLD-0137              | Invalid token, please provide appropriate token to call the API                                                                                  |
| SIG-VLD-0138              | You are not authorized to perform the current operation                                                                                          |
| SIG-VLD-0139              | Token parser error, invalid token. Please send valid token.                                                                                      |
| SIG-VLD-0140              | Identifier value `%s`, cannot contain \<,>,",”,= characters.                                                                                     |
| SIG-VLD-0141              | Region cannot contain \<,>,",”,= characters.                                                                                                     |
| SIG-VLD-0142              | Datasource cannot contain \<,>,",”,= characters.                                                                                                 |
| SIG-VLD-0143              | Comments cannot contain \<,>,",”,= characters.                                                                                                   |
| SIG-VLD-0144              | Logged in Company does not match with task initiator company                                                                                     |
| SIG-VLD-0145              | Assigning user should have `%s`.                                                                                                                 |
| SIG-VLD-0146              | Material Category Name must not be null or empty.                                                                                                |
| SIG-VLD-0147              | Invalid Material Category Id (UUID).                                                                                                             |
| SIG-VLD-0148              | Material Category image filename is limited to 100 characters                                                                                    |
| SIG-VLD-0149              | Invalid image file                                                                                                                               |
| SIG-VLD-0150              | Invalid image file content Type                                                                                                                  |
| SIG-VLD-0151              | Weight must be greater than zero.                                                                                                                |
| SIG-VLD-0152              | Revision number length should not be more than 100.                                                                                              |
| SIG-VLD-0153              | Product Carbon footprint must not be null or empty.                                                                                              |
| SIG-VLD-0154              | Factory id must not be null or empty.                                                                                                            |
| SIG-VLD-0155              | Invalid factory id (UUID).                                                                                                                       |
| SIG-VLD-0156              | Product carbon footprint value must not be less than 0.                                                                                          |
| SIG-VLD-0157              | Comment must be less than or equal to 250 characters.                                                                                            |
| SIG-VLD-0158              | Source system must be less than or equal to 200 characters.                                                                                      |
| SIG-VLD-0159              | Emission data must not be empty.                                                                                                                 |
| SIG-VLD-0160              | Factory emissions data must not be empty.                                                                                                        |
| SIG-VLD-0161              | Maximum allowed factory emissions data size exceeded.                                                                                            |
| SIG-VLD-0162              | Batch number must not be null or empty.                                                                                                          |
| SIG-VLD-0163              | Batch number must be less than or equal to 100 characters.                                                                                       |
| SIG-VLD-0164              | Data source must not be null or empty.                                                                                                           |
| SIG-VLD-0165              | Data source must be less than or equal to 100 characters.                                                                                        |
| SIG-VLD-0166              | Comment must be less than or equal to 100 characters.                                                                                            |
| SIG-VLD-0167              | Quantity must be greater than zero.                                                                                                              |
| SIG-VLD-0168              | Type of activity must not be null or empty.                                                                                                      |
| SIG-VLD-0169              | Type of activity must be less than or equal to 50 characters.                                                                                    |
| SIG-VLD-0170              | Primary share in emission data must not be empty.                                                                                                |
| SIG-VLD-0171              | Total in emission data must not be empty.                                                                                                        |
| SIG-VLD-0172              | Comment must be less than or equal to 100 characters.                                                                                            |
| SIG-VLD-0173              | Total share in emission data must not be empty.                                                                                                  |
| SIG-VLD-0174              | Invalid emission unit. Emission unit should be valid unit.                                                                                       |
| SIG-VLD-0175              | Supplier name cannot contain \<,>,",”,= characters.                                                                                              |
| SIG-VLD-0176              | Task is not eligible for `%s` with `%s` status.                                                                                                  |
| SIG-VLD-0177              | Task with the given ID `{}` does not belong to the company with ID `{}`                                                                          |
| SIG-VLD-0178              | Invalid task status for `%s`.                                                                                                                    |
| SIG-VLD-0179              | PCFData Id (UUID) must not be null or empty                                                                                                      |
| SIG-VLD-0180              | Invalid PCFData Id (UUID).                                                                                                                       |
| SIG-VLD-0181              | Invalid Network type selected                                                                                                                    |
| SIG-VLD-0182              | This endpoint is not implemented. Please use `%s`/events endpoint instead.                                                                       |
| SIG-VLD-0183              | Company counter already exists.                                                                                                                  |
| SIG-VLD-0184              | Company missing counter.                                                                                                                         |
| SIG-VLD-0185              | Choose a family name other than Uncategorized.                                                                                                   |
| SIG-VLD-0186              | Active emission must not be null or empty                                                                                                        |
| SIG-VLD-0187              | Invalid active emission.                                                                                                                         |
| SIG-VLD-0188              | Mismatch between provided task IDs `%s` and found tasks.                                                                                         |
| SIG-VLD-0189              | Given limit is invalid                                                                                                                           |
| SIG-VLD-0190              | Task company counter is missing                                                                                                                  |
| SIG-VLD-0191              | Invalid BPNL, please provide a valid BPNL.                                                                                                       |
| SIG-VLD-0192              | BPNL is already assigned to another company.                                                                                                     |
| SIG-VLD-0193              | Invalid EDC URL.                                                                                                                                 |
| SIG-VLD-0194              | Company is already activated with the given provider.                                                                                            |
| SIG-VLD-0195              | Duplicate property name.                                                                                                                         |
| SIG-VLD-0196              | Invalid property name, Please provide valid property name.                                                                                       |
| SIG-VLD-0197              | Invalid command, Please provide valid command.                                                                                                   |
| SIG-VLD-0198              | Invalid network provider, Please provide valid network provider.                                                                                 |
| SIG-VLD-0199              | Status cannot be null or empty.                                                                                                                  |
| SIG-VLD-0200              | Network provider name cannot be null or empty.                                                                                                   |
| SIG-VLD-0201              | Property name cannot be null or empty.                                                                                                           |
| SIG-VLD-0202              | Property `%s` exceeds maximum length `%s`                                                                                                        |
| SIG-VLD-0203              | Value cannot be null or empty for `%s`.                                                                                                          |
| SIG-VLD-0204              | Network must have at least one connecting network properties                                                                                     |
| SIG-VLD-0205              | Property Value must be less than or equal to 50 characters.                                                                                      |
| SIG-VLD-0206              | Company Identifier Type Id (UUID) must not be null or empty.                                                                                     |
| SIG-VLD-0207              | Failed to parse credential data.                                                                                                                 |
| SIG-VLD-0208              | Invalid product family id (UUID).                                                                                                                |
| SIG-VLD-0209              | Cannot Deactivate, Company Network must be Activated first.                                                                                      |
| SIG-VLD-0210              | Duplicate network name.                                                                                                                          |
| SIG-VLD-0211              | Reason is required to Deactivate.                                                                                                                |
| SIG-VLD-0212              | Inbound Catena response is empty.                                                                                                                |
| SIG-VLD-0213              | PCF Data in Inbound Catena response is empty.                                                                                                    |
| SIG-VLD-0214              | Inbound Catena response - Company Name is Mandatory.                                                                                             |
| SIG-VLD-0215              | Inbound Catena response - Company Id is Mandatory.                                                                                               |
| SIG-VLD-0216              | Inbound Catena response - Product Name is Mandatory.                                                                                             |
| SIG-VLD-0217              | Inbound Catena response - Product Id is Mandatory.                                                                                               |
| SIG-VLD-0218              | Inbound Catena response - Declared Unit is Mandatory.                                                                                            |
| SIG-VLD-0219              | Inbound Catena response - Unitary Product Amount is Mandatory.                                                                                   |
| SIG-VLD-0220              | Inbound Catena response - Created Date is Mandatory.                                                                                             |
| SIG-VLD-0221              | Inbound Catena response - Cross Sectoral Standards is Mandatory.                                                                                 |
| SIG-VLD-0222              | Inbound Catena response - Sources Secondary Data is Mandatory.                                                                                   |
| SIG-VLD-0223              | Inbound Catena response - gwpTotal is Mandatory.                                                                                                 |
| SIG-VLD-0224              | Inbound Catena response - Product mass per declared unit is Mandatory.                                                                           |
| SIG-VLD-0225              | Inbound Catena response - Geography with region or subregion is Mandatory.                                                                       |
| SIG-VLD-0226              | Inbound Catena response - Product or sector rules is Mandatory.                                                                                  |
| SIG-VLD-0227              | Inbound Catena response - PCF ID is Mandatory.                                                                                                   |
| SIG-VLD-0228              | Inbound Catena response - Reference Period Start is Mandatory.                                                                                   |
| SIG-VLD-0229              | Inbound Catena response - Reference Period End is Mandatory.                                                                                     |
| SIG-VLD-0230              | Inbound Catena response - IPCC Report Version of GWP Value is Mandatory.                                                                         |
| SIG-VLD-0231              | CUSTOMER_BPNL is mandatory to share the pcf data to catena-x.                                                                                    |
| SIG-VLD-0232              | Invalid lifecycle type name provided, please provide a valid life cycle type name.                                                               |
| SIG-VLD-0233              | Company Id (UUID) must be less than or equal to 36 characters.                                                                                   |
| SIG-VLD-0234              | Company Id is required.                                                                                                                          |
| SIG-VLD-0235              | Invalid Company Id. Company Id must be Valid UUID.                                                                                               |
| SIG-VLD-0236              | For `%s`, `%s` must not be null or empty.                                                                                                        |
| SIG-VLD-0237              | `%s` is invalid.                                                                                                                                 |
| SIG-VLD-0238              | SUPPLIER_BPNL is mandatory to request the pcf data from catena-x.                                                                                |
| SIG-VLD-0239              | Invalid Pcf State                                                                                                                                |
| SIG-VLD-0240              | Invalid Company Identifier Type Id (UUID)                                                                                                        |
| SIG-VLD-0241              | Company Identifier Type Id (UUID) must be less than or equal to 36 characters.                                                                   |
| SIG-VLD-0242              | Task Id (UUID) must be less than or equal to 36 characters.                                                                                      |
| SIG-VLD-0243              | Task Id (UUID) must not be null or empty                                                                                                         |
| SIG-VLD-0244              | Invalid Task Id (UUID).                                                                                                                          |
| SIG-VLD-0245              | Credential Id must be less than or equal to 36 characters.                                                                                       |
| SIG-VLD-0246              | Credential Id must not be null or empty                                                                                                          |
| SIG-VLD-0247              | Invalid Credential Id.                                                                                                                           |
| SIG-VLD-0248              | Share Type cannot be null or empty.                                                                                                              |
| SIG-VLD-0249              | Product Id (UUID) must be less than or equal to 36 characters.                                                                                   |
| SIG-VLD-0250              | Product Id (UUID) must not be null or empty                                                                                                      |
| SIG-VLD-0251              | Invalid Product Id (UUID).                                                                                                                       |
| SIG-VLD-0252              | Invalid network provider type, Please provide valid network provider type.                                                                       |
| SIG-VLD-0253              | Property `%s` should be present in the request.                                                                                                  |
| SIG-VLD-0254              | BOM version cannot be null.                                                                                                                      |
| SIG-VLD-0255              | BOM Version must be less than or equal to 50 characters.                                                                                         |
| SIG-VLD-0256              | Comment must be less than or equal to 200 characters.                                                                                            |
| SIG-VLD-0257              | Revision number must be less than equal to 200 characters.                                                                                       |
| SIG-VLD-0258              | BOM data should not be empty.                                                                                                                    |
| SIG-VLD-0259              | Bom nested levels cannot be more than `%s`                                                                                                       |
| SIG-VLD-0260              | Component Identifier value must not be null or empty.                                                                                            |
| SIG-VLD-0261              | Component Identifier value must be less than or equal to 100 characters.                                                                         |
| SIG-VLD-0262              | Group name must not be null or empty.                                                                                                            |
| SIG-VLD-0263              | Group name must be less than or equal to 50 characters.                                                                                          |
| SIG-VLD-0264              | Group Identifier value must not be null or empty.                                                                                                |
| SIG-VLD-0265              | BOM should not be empty.                                                                                                                         |
| SIG-VLD-0266              | Invalid Date Format. Please send valid date format                                                                                               |
| SIG-VLD-0267              | Error while parsing component emission proof details.                                                                                            |
| SIG-VLD-0268              | Error while parsing proof json.                                                                                                                  |
| SIG-VLD-0269              | No proof found for given component supplier link.                                                                                                |
| SIG-VLD-0270              | Proof is not valid.                                                                                                                              |
| SIG-VLD-0271              | Cannot update, company network must be activated first.                                                                                          |
| SIG-VLD-0272              | Invalid request payload.                                                                                                                         |
| SIG-VLD-0273              | `%s` is empty.                                                                                                                                   |
| SIG-VLD-0274              | Invalid file type `%s`.                                                                                                                          |
| SIG-VLD-0275              | File does not exist.                                                                                                                             |
| SIG-VLD-0276              | Duplicate file `%s`, please only upload file with unique names.                                                                                  |
| SIG-VLD-0277              | Invalid date format. Please send valid date format.                                                                                              |
| SIG-VLD-0278              | Filename `%s`, cannot be more than `%s` characters.                                                                                              |
| SIG-VLD-0279              | From date cannot be greater than To date                                                                                                         |
| SIG-VLD-0280              | String.format("Invalid status type, supported status types are `%s`", String.join(",", JobStatusTypeEnum.ALL))                                   |
| SIG-VLD-0281              | To date is missing                                                                                                                               |
| SIG-VLD-0282              | From date is missing                                                                                                                             |
| SIG-VLD-0283              | A maximum of 20 suggestions can be requested.                                                                                                    |
| SIG-VLD-0284              | Company Identifier value can not be null or empty.                                                                                               |
| SIG-VLD-0285              | Company Identifier value must be less than or equal to 100 characters.                                                                           |
| SIG-VLD-0286              | Error occurred while parsing the PCF mass upload file.                                                                                           |
| SIG-VLD-0287              | Company Identifier discoverable flag can not be null                                                                                             |
| SIG-VLD-0288              | Empty excel cannot be uploaded.                                                                                                                  |
| SIG-VLD-0289              | Only `%s` rows are allowed.                                                                                                                      |
| SIG-VLD-0290              | Upload only excel file                                                                                                                           |
| SIG-VLD-0291              | Unknown standard.                                                                                                                                |
| SIG-VLD-0292              | File is malicious or corrupted.                                                                                                                  |
| SIG-VLD-0293              | Error occurred while file scan.                                                                                                                  |
| SIG-VLD-0294              | Unknown declaration type.                                                                                                                        |
| SIG-VLD-0295              | Unknown life cycle type.                                                                                                                         |
| SIG-VLD-0296              | Maximum upload size exceeded, It should be under `%s`.                                                                                           |
| SIG-VLD-0297              | Invalid file type.                                                                                                                               |
| SIG-VLD-0298              | File has no data.                                                                                                                                |
| SIG-VLD-0299              | Unknown company factory.                                                                                                                         |
| SIG-VLD-0300              | Maximum 10 company identifiers are allowed to save at a time                                                                                     |
| SIG-VLD-0301              | Company does not exist                                                                                                                           |
| SIG-VLD-0302              | User does not exist                                                                                                                              |
| SIG-VLD-0303              | Invalid product declared unit                                                                                                                    |
| SIG-VLD-0304              | Invalid product identifiers                                                                                                                      |
| SIG-VLD-0305              | Invalid main product identifier (length >100)                                                                                                    |
| SIG-VLD-0306              | Invalid product name (length>100)                                                                                                                |
| SIG-VLD-0307              | Invalid quantity value (length >15)                                                                                                              |
| SIG-VLD-0308              | Invalid weight (length>15 )                                                                                                                      |
| SIG-VLD-0309              | Invalid numeric data in weight                                                                                                                   |
| SIG-VLD-0310              | Invalid numeric data in quantity                                                                                                                 |
| SIG-VLD-0311              | Invalid company identifier type found in request                                                                                                 |
| SIG-VLD-0312              | Unable to link location                                                                                                                          |
| SIG-VLD-0313              | Invalid date entered. Please enter a valid date.                                                                                                 |
| SIG-VLD-0314              | Product with same identifier exists                                                                                                              |
| SIG-VLD-0315              | Could not find valid entries in file                                                                                                             |
| SIG-VLD-0316              | Uploaded file template is invalid, use the latest template                                                                                       |
| SIG-VLD-0317              | Requested page number value is invalid.                                                                                                          |
| SIG-VLD-0318              | Requested page size value is invalid.                                                                                                            |
| SIG-VLD-0319              | PageSize cannot Exceed allowed Limit `%s`                                                                                                        |
| SIG-VLD-0320              | Field `%s` cannot be null                                                                                                                        |
| SIG-VLD-0321              | Mass uploading functionality is available only for TFS member                                                                                    |
| SIG-VLD-0322              | Company contact phone number is in incorrect format                                                                                              |
| SIG-VLD-0323              | Product with the given ID `%s` does not exist                                                                                                    |
| SIG-VLD-0324              | Draft with the given ID `%s` does not exist                                                                                                      |
| SIG-VLD-0325              | Missing/Incorrect Product Identifier                                                                                                             |
| SIG-VLD-0326              | Draft does not exists or deleted                                                                                                                 |
| SIG-VLD-0327              | Invalid target scope selection                                                                                                                   |
| SIG-VLD-0328              | Total incl./excl. bio can have only numeric data                                                                                                 |
| SIG-VLD-0329              | Miss match product details                                                                                                                       |
| SIG-VLD-0330              | Contact email address is in incorrect format                                                                                                     |
| SIG-VLD-0331              | File name exceeds max allowed allowed length of %d characters. Please rename the file and upload again.                                          |
| SIG-VLD-0332              | Invalid product identifiers `%s`. Product identifiers cannot contain \<,>,",”,= characters.                                                      |
| SIG-VLD-0333              | Could not find the draft                                                                                                                         |
| SIG-VLD-0334              | This is not a valid draft                                                                                                                        |
| SIG-VLD-0335              | Product name cannot contain \<,>,",”,= characters.                                                                                               |
| SIG-VLD-0336              | Product name must not be null or empty.                                                                                                          |
| SIG-VLD-0337              | Invalid description (length>500)                                                                                                                 |
| SIG-VLD-0338              | Product description cannot contain \<,>,",”,= characters.                                                                                        |
| SIG-VLD-0339              | Invalid product family name (length>100 )                                                                                                        |
| SIG-VLD-0340              | Product family name cannot contain \<,>,",”,= characters.                                                                                        |
| SIG-VLD-0341              | Client Id must not be null or empty.                                                                                                             |
| SIG-VLD-0342              | Company Id must not be null or empty.                                                                                                            |
| SIG-VLD-0343              | Company Id cannot exceed 200 characters.                                                                                                         |
| SIG-VLD-0344              | Product Id must not be null or empty.                                                                                                            |
| SIG-VLD-0345              | Product Code CAS cannot exceed 20 characters.                                                                                                    |
| SIG-VLD-0346              | Product Description cannot exceed 2000 characters.                                                                                               |
| SIG-VLD-0347              | Declared Unit must not be null or empty.                                                                                                         |
| SIG-VLD-0348              | Invalid Declared Unit.                                                                                                                           |
| SIG-VLD-0349              | Unitary Product Amount must be a number greater than equals to 0.                                                                                |
| SIG-VLD-0350              | Unitary Product Amount cannot exceed 20 characters.                                                                                              |
| SIG-VLD-0351              | Preceding Pf Id cannot exceed 400 characters.                                                                                                    |
| SIG-VLD-0352              | Exempted Emissions Percent is required.                                                                                                          |
| SIG-VLD-0353              | Exempted Emissions Percent must be a number between 0 and 10.                                                                                    |
| SIG-VLD-0354              | Exempted Emissions Percent cannot exceed 15 characters.                                                                                          |
| SIG-VLD-0355              | Exempted Emissions Description cannot exceed 300 characters.                                                                                     |
| SIG-VLD-0356              | Invalid Recycled Content Type.                                                                                                                   |
| SIG-VLD-0357              | CCU Co2 Origin cannot exceed 1000 characters.                                                                                                    |
| SIG-VLD-0358              | Invalid Geography Country Subdivision.                                                                                                           |
| SIG-VLD-0359              | Invalid Geography Country.                                                                                                                       |
| SIG-VLD-0360              | Geography Region/Subregion must not be null or empty.                                                                                            |
| SIG-VLD-0361              | Invalid Geography Region/Subregion.                                                                                                              |
| SIG-VLD-0362              | Invalid Product Handover Location.                                                                                                               |
| SIG-VLD-0363              | Reference Period Start must not be null or empty.                                                                                                |
| SIG-VLD-0364              | Reference Period Start must be a valid date in present or past(Eg: 2024-02-01T08:30:00.000Z).                                                    |
| SIG-VLD-0365              | Reference Period End must not be null or empty.                                                                                                  |
| SIG-VLD-0366              | Reference Period End must be a valid date in present or past(Eg: 2024-02-01T08:30:00.000Z).                                                      |
| SIG-VLD-0367              | Created must not be null or empty.                                                                                                               |
| SIG-VLD-0368              | Created must be a valid date in present or past(Eg: 2024-02-01T08:30:00.000Z).                                                                   |
| SIG-VLD-0369              | Validity Period Start must be a valid date(Eg: 2024-02-01T08:30:00.000Z).                                                                        |
| SIG-VLD-0370              | Validity Period End must be a valid date(Eg: 2024-02-01T08:30:00.000Z).                                                                          |
| SIG-VLD-0371              | Cross Sectoral Standards Used must not be null or empty.                                                                                         |
| SIG-VLD-0372              | Cross Sectoral Standards Used cannot exceed 200 characters.                                                                                      |
| SIG-VLD-0373              | Invalid Cross Sectoral Standards Used.                                                                                                           |
| SIG-VLD-0374              | Product/Sector Specific Rules must not be null or empty.                                                                                         |
| SIG-VLD-0375              | Product/Sector Specific Rules cannot exceed 200 characters.                                                                                      |
| SIG-VLD-0376              | Invalid Product/Sector Specific Rules.                                                                                                           |
| SIG-VLD-0377              | Allocation Rules Description must not be null or empty.                                                                                          |
| SIG-VLD-0378              | Allocation Rules Description cannot exceed 1000 characters.                                                                                      |
| SIG-VLD-0379              | Allocation Waste Incineration must not be null or empty.                                                                                         |
| SIG-VLD-0380              | Invalid Allocation Waste Incineration.                                                                                                           |
| SIG-VLD-0381              | Invalid Allocation Recycled Carbon.                                                                                                              |
| SIG-VLD-0382              | Allocation CCU Carbon cannot exceed 1000 characters.                                                                                             |
| SIG-VLD-0383              | Primary Data Share must be a number between 0 and 100.                                                                                           |
| SIG-VLD-0384              | Primary Data Share cannot exceed 15 characters.                                                                                                  |
| SIG-VLD-0385              | Secondary Emission Factor Sources must not be null or empty.                                                                                     |
| SIG-VLD-0386              | Secondary Emission Factor Sources cannot exceed 200 characters.                                                                                  |
| SIG-VLD-0387              | Comment cannot exceed 10000 characters.                                                                                                          |
| SIG-VLD-0388              | Technological DQR must be a number between 1 and 3.                                                                                              |
| SIG-VLD-0389              | Technological DQR cannot exceed 15 characters.                                                                                                   |
| SIG-VLD-0390              | Temporal DQR must be a number between 1 and 3.                                                                                                   |
| SIG-VLD-0391              | Temporal DQR cannot exceed 15 characters.                                                                                                        |
| SIG-VLD-0392              | Geographical DQR must be a number between 1 and 3.                                                                                               |
| SIG-VLD-0393              | Geographical DQR cannot exceed 15 characters.                                                                                                    |
| SIG-VLD-0394              | Completeness DQR must be a number between 1 and 3.                                                                                               |
| SIG-VLD-0395              | Completeness DQR cannot exceed 15 characters.                                                                                                    |
| SIG-VLD-0396              | Reliability DQR must be a number between 1 and 3.                                                                                                |
| SIG-VLD-0397              | Reliability DQR cannot exceed 15 characters.                                                                                                     |
| SIG-VLD-0398              | PCF Including Biogenic cannot exceed 15 characters.                                                                                              |
| SIG-VLD-0399              | PCF Including Biogenic must be a valid number.                                                                                                   |
| SIG-VLD-0400              | PCF Excluding Biogenic cannot exceed 15 characters.                                                                                              |
| SIG-VLD-0401              | PCF Excluding Biogenic must be a valid number.                                                                                                   |
| SIG-VLD-0402              | Fossil GHG Emissions cannot exceed 15 characters.                                                                                                |
| SIG-VLD-0403              | Fossil GHG Emissions must be a valid number.                                                                                                     |
| SIG-VLD-0404              | Biogenic Carbon Emissions Other Than CO2 cannot exceed 15 characters.                                                                            |
| SIG-VLD-0405              | Biogenic Carbon Emissions Other Than CO2 must be a valid number.                                                                                 |
| SIG-VLD-0406              | Biogenic Carbon Withdrawal cannot exceed 15 characters.                                                                                          |
| SIG-VLD-0407              | Biogenic Carbon Withdrawal must be a valid number.                                                                                               |
| SIG-VLD-0408              | D LUC GHG Emissions cannot exceed 15 characters.                                                                                                 |
| SIG-VLD-0409              | D LUC GHG Emissions must be a valid number.                                                                                                      |
| SIG-VLD-0410              | LU GHG Emissions cannot exceed 15 characters.                                                                                                    |
| SIG-VLD-0411              | LU GHG Emissions must be a valid number.                                                                                                         |
| SIG-VLD-0412              | Aircraft GHG Emissions cannot exceed 15 characters.                                                                                              |
| SIG-VLD-0413              | Aircraft GHG Emissions must be a valid number.                                                                                                   |
| SIG-VLD-0414              | Total Carbon Content cannot exceed 15 characters.                                                                                                |
| SIG-VLD-0415              | Total Carbon Content must be a valid number.                                                                                                     |
| SIG-VLD-0416              | Biogenic Carbon Content cannot exceed 15 characters.                                                                                             |
| SIG-VLD-0417              | Biogenic Carbon Content must be a valid number.                                                                                                  |
| SIG-VLD-0418              | Recycled Carbon Content cannot exceed 15 characters.                                                                                             |
| SIG-VLD-0419              | Recycled Carbon Content must be a valid number.                                                                                                  |
| SIG-VLD-0420              | CCU Based Carbon Content cannot exceed 15 characters.                                                                                            |
| SIG-VLD-0421              | CCU Based Carbon Content must be a valid number.                                                                                                 |
| SIG-VLD-0422              | Product Mass Per Declaration Unit cannot exceed 15 characters.                                                                                   |
| SIG-VLD-0423              | Product Mass Per Declaration Unit must be a valid number.                                                                                        |
| SIG-VLD-0424              | Boundary Processes Description cannot exceed 2000 characters.                                                                                    |
| SIG-VLD-0425              | PCF Excluding Biogenic must not be null or empty.                                                                                                |
| SIG-VLD-0426              | Invalid product id, please provide valid product identifier to create PCF.                                                                       |
| SIG-VLD-0427              | Invalid product identifiers passed in the payload - `%s`, please provide valid product identifiers.                                              |
| SIG-VLD-0428              | For cross sectoral standard, only one other value is allowed apart from the standard list, but input contains more than one - `%s`.              |
| SIG-VLD-0429              | Company Field is required.                                                                                                                       |
| SIG-VLD-0430              | Product Field is required.                                                                                                                       |
| SIG-VLD-0431              | PCFData Field is required.                                                                                                                       |
| SIG-VLD-0432              | Partial Full PCF is required.                                                                                                                    |
| SIG-VLD-0433              | Invalid Partial Full PCF.                                                                                                                        |
| SIG-VLD-0434              | Packaging Emissions Included is required.                                                                                                        |
| SIG-VLD-0435              | Invalid Packaging Emissions Included.                                                                                                            |
| SIG-VLD-0436              | Scope PCF is required.                                                                                                                           |
| SIG-VLD-0437              | Production Stage is required.                                                                                                                    |
| SIG-VLD-0438              | Mandatory input `%s` is missing.                                                                                                                 |
| SIG-VLD-0439              | Internal server error. Please retry again.                                                                                                       |
| SIG-VLD-0440              | Invalid Job Type.                                                                                                                                |
| SIG-VLD-0441              | Request Payload has Duplicate ProductIds                                                                                                         |
| SIG-VLD-0442              | Unitary Product Amount is required.                                                                                                              |
| SIG-VLD-0443              | Product Mass Per Declaration Unit is required.                                                                                                   |
| SIG-VLD-0444              | Reference Period End must be greater than or equals to Reference Period Start                                                                    |
| SIG-VLD-0445              | Validity Period End is required.                                                                                                                 |
| SIG-VLD-0446              | Validity Period End must be greater than or equals to Validity Period Start                                                                      |
| SIG-VLD-0447              | Carbon Content Total is required.                                                                                                                |
| SIG-VLD-0448              | Characterization Factors must not be null or empty.                                                                                              |
| SIG-VLD-0449              | Invalid Characterization Factors.                                                                                                                |
| SIG-VLD-0450              | Product carbon content is required.                                                                                                              |
| SIG-VLD-0451              | Future date entered. Please enter a date that is today or in the past.                                                                           |
| SIG-VLD-0452              | Job Item is not available for jobItemUid: {0}.                                                                                                   |
| SIG-VLD-0453              | Product Emission Credential not available for jobItemUid: {0}, CompanyUid: {1}.                                                                  |
| SIG-VLD-0454              | File {0} has no data.                                                                                                                            |
| SIG-VLD-0455              | Link/Unlink value must not be null or empty                                                                                                      |
| SIG-VLD-0456              | Invalid Link/Unlink value                                                                                                                        |
| SIG-VLD-0457              | Cannot accept revoked credential.                                                                                                                |
| SIG-VLD-0458              | Product identifier is mandatory to request pcf.                                                                                                  |
| SIG-VLD-0459              | Invalid pcf Standard.                                                                                                                            |
| SIG-VLD-0460              | Request payload cannot be more than %dMB.                                                                                                        |
| SIG-VLD-0461              | Invalid Contact Number.                                                                                                                          |
| SIG-VLD-0462              | Cannot Update. Company Network must be requested first.                                                                                          |
| SIG-VLD-0463              | Update is not allowed as the request is already processed.                                                                                       |
| SIG-VLD-0464              | Invalid Request Message                                                                                                                          |
| SIG-VLD-0465              | User Id (UUID) must be less than or equal to 36 characters.                                                                                      |
| SIG-VLD-0466              | Invalid User Id (UUID).                                                                                                                          |
| SIG-VLD-0467              | Invalid numeric data `%s`                                                                                                                        |
| SIG-VLD-0468              | Cannot Deny, Company Network must be in requested state.                                                                                         |
| SIG-VLD-0469              | Reason is required to Deny.                                                                                                                      |
| SIG-VLD-0470              | Invalid supplier status                                                                                                                          |
| SIG-VLD-0471              | Request cannot be cancelled as the system admin has already {0} the request.                                                                     |
| SIG-VLD-0472              | There is no request to cancel.                                                                                                                   |
| SIG-VLD-0473              | Domain cannot be null or empty.                                                                                                                  |
| SIG-VLD-0474              | Text cannot be null or empty.                                                                                                                    |
| SIG-VLD-0475              | A minimum of 1 characters required in text field.                                                                                                |
| SIG-VLD-0476              | A maximum of 100 characters can be text field.                                                                                                   |
| SIG-VLD-0477              | Years before 1990 are not supported.                                                                                                             |
| SIG-VLD-0478              | Parameters cannot be null                                                                                                                        |
| SIG-VLD-0479              | Invalid request. `%s`                                                                                                                            |
| SIG-VLD-0480              | Invalid Domain Variant                                                                                                                           |
| SIG-VLD-0481              | Invalid Weight Units                                                                                                                             |
| SIG-VLD-0482              | Invalid Volume Units                                                                                                                             |
| SIG-VLD-0483              | Estimation Request cannot be null                                                                                                                |
| SIG-VLD-0484              | Invalid Parameter                                                                                                                                |
| SIG-VLD-0485              | Max suggestions value needs to be at least one.                                                                                                  |
| SIG-VLD-0486              | Text cannot contain \<,>,",”,= characters.                                                                                                       |
| SIG-VLD-0487              | Search text must be between 3 and 50 characters.                                                                                                 |
| SIG-VLD-0488              | Invalid weight.                                                                                                                                  |
| SIG-VLD-0489              | Invalid volume.                                                                                                                                  |
| SIG-VLD-0490              | Mismatch between provided requested task unit `%s` and credential unit.                                                                          |
| SIG-VLD-0491              | Pcf Id must be less than or equal to 36 characters.                                                                                              |
| SIG-VLD-0492              | Pcf Id must not be null or empty                                                                                                                 |
| SIG-VLD-0493              | Invalid PCF Id.                                                                                                                                  |
| SIG-VLD-0494              | Task share not possible as product not linked                                                                                                    |
| SIG-VLD-0495              | Reason is required to Reject.                                                                                                                    |
| SIG-VLD-0496              | Invalid number of quantities                                                                                                                     |
| SIG-VLD-0497              | Suggestion Id cannot be null or empty.                                                                                                           |
| SIG-VLD-0498              | SpecVersion is required.                                                                                                                         |
| SIG-VLD-0499              | Invalid Partial Full PCF.                                                                                                                        |
| SIG-VLD-0500              | Packaging GHG Emissions cannot exceed 15 characters.                                                                                             |
| SIG-VLD-0501              | Packaging GHG Emissions must be a valid number.                                                                                                  |
| SIG-VLD-0502              | PCF Legal Statement cannot exceed 10000 characters.                                                                                              |
| SIG-VLD-0503              | Unknown spec Version: `%s`                                                                                                                       |
| SIG-VLD-0504              | Invalid Product Id (UUID).                                                                                                                       |
| SIG-VLD-0505              | From date and To date are missing                                                                                                                |
| SIG-VLD-0506              | Biogenic Emissions cannot exceed 15 characters.                                                                                                  |
| SIG-VLD-0507              | Biogenic Emissions must be a valid number.                                                                                                       |
| SIG-VLD-0508              | dLuc and LuGhg Emissions cannot exceed 15 characters.                                                                                            |
| SIG-VLD-0509              | dLuc and LuGhg Emissions must be a valid number.                                                                                                 |
| SIG-VLD-0510              | iLucGhg Emissions cannot exceed 15 characters.                                                                                                   |
| SIG-VLD-0511              | iLucGhg Emissions must be a valid number.                                                                                                        |
| SIG-VLD-0512              | land Management Ghg Emissions cannot exceed 15 characters.                                                                                       |
| SIG-VLD-0513              | land Management Ghg Emissions must be a valid number.                                                                                            |
| SIG-VLD-0514              | other Biogenic Ghg Emissions cannot exceed 15 characters.                                                                                        |
| SIG-VLD-0515              | other Biogenic Ghg Emissions must be a valid number.                                                                                             |
| SIG-VLD-0516              | Invalid From Date Format. Please send valid date format (for eg: "")                                                                             |
| SIG-VLD-0517              | Invalid To Date Format. Please send valid date format (for eg: "")                                                                               |
| SIG-VLD-0518              | Invalid plan                                                                                                                                     |
| SIG-VLD-0519              | `%s` plan is already enabled for the company                                                                                                     |
| SIG-VLD-0520              | Invalid type                                                                                                                                     |
| SIG-VLD-0521              | Name of the user should be 100 characters                                                                                                        |
| SIG-VLD-0522              | Customer contact number is limited to 30 character                                                                                               |
| SIG-VLD-0523              | Subject is limited to 200 character                                                                                                              |
| SIG-VLD-0524              | Add on is limited to 100 character                                                                                                               |
| SIG-VLD-0525              | Plan is limited to 100 character                                                                                                                 |
| SIG-VLD-0526              | Message is limited to `%s` characters                                                                                                            |
| SIG-VLD-0527              | Invalid component category                                                                                                                       |
| SIG-VLD-0528              | Product Description should not be null or empty.                                                                                                 |
| SIG-VLD-0529              | Production stage data cant be null                                                                                                               |
| SIG-VLD-0530              | Invalid factory emission UUID                                                                                                                    |
| SIG-VLD-0531              | Total Incl. bio or Total excl. bio should not be null or empty                                                                                   |
| SIG-VLD-0532              | Product Code CPC cannot exceed 20 characters.                                                                                                    |
| SIG-VLD-0533              | Unknown `%s` : `%s`                                                                                                                              |
| SIG-VLD-0534              | Identifier value cannot contain \<,>,",”,= characters.                                                                                           |
| SIG-VLD-0535              | Message cannot contain \<,>,",”,= characters.                                                                                                    |
| SIG-VLD-0536              | Supplier Product Id cannot contain \<,>,",”,= characters.                                                                                        |
| SIG-VLD-0537              | An Access Group with this name already exists.                                                                                                   |
| SIG-VLD-0538              | Company Identifier not found.                                                                                                                    |
| SIG-VLD-0539              | Company Identifier Id (UUID) must be less than or equal to 36 characters.                                                                        |
| SIG-VLD-0540              | Company Identifier Id (UUID). must not be null or empty                                                                                          |
| SIG-VLD-0541              | Invalid Company Identifier Id (UUID).                                                                                                            |
| SIG-VLD-0542              | Email cannot be more than 100 characters.                                                                                                        |
| SIG-VLD-0543              | Contact Phone number cannot be more than 20 characters.                                                                                          |
| SIG-VLD-0544              | Production stage included data cant be null.                                                                                                     |
| SIG-VLD-0545              | Invalid Production stage Included.                                                                                                               |
| SIG-VLD-0546              | Invalid Material acquisition and pre-processing sub-stage Included.                                                                              |
| SIG-VLD-0547              | Invalid Raw material acquisition sub-stage Included.                                                                                             |
| SIG-VLD-0548              | Invalid Transport to manufacturing site sub-stage Included.                                                                                      |
| SIG-VLD-0549              | Invalid Pre-processing sub-stage Included.                                                                                                       |
| SIG-VLD-0550              | Invalid Manufacturing sub-stage Included.                                                                                                        |
| SIG-VLD-0551              | Partial Full PCF must not be null or empty.                                                                                                      |
| SIG-VLD-0552              | Invalid Assembly sub-stage Included.                                                                                                             |
| SIG-VLD-0553              | Invalid End-of-life treatment of generated waste from manufacturing sub-stage 2 Included.                                                        |
| SIG-VLD-0554              | Invalid Distribution stage Included.                                                                                                             |
| SIG-VLD-0555              | Invalid Transport A to distributer sub-stage Included.                                                                                           |
| SIG-VLD-0556              | Invalid Reconditioning at distributor sub-stage Included.                                                                                        |
| SIG-VLD-0557              | Invalid Transport B to place of use sub-stage Included.                                                                                          |
| SIG-VLD-0558              | Invalid Installation stage Included.                                                                                                             |
| SIG-VLD-0559              | Invalid Installation sub-stage Included.                                                                                                         |
| SIG-VLD-0560              | Invalid End-of-life treatment of generated waste from installation sub-stage Included.                                                           |
| SIG-VLD-0561              | Invalid Usage stage Included.                                                                                                                    |
| SIG-VLD-0562              | Invalid Use sub-stage Included.                                                                                                                  |
| SIG-VLD-0563              | Invalid Maintenance sub-stage Included.                                                                                                          |
| SIG-VLD-0564              | Invalid Repair sub-stage Included.                                                                                                               |
| SIG-VLD-0565              | Invalid Replacement sub-stage Included.                                                                                                          |
| SIG-VLD-0566              | Invalid End-of-life treatment of generated waste from use sub-stage Included.                                                                    |
| SIG-VLD-0567              | Invalid Refurbishment sub-stage Included.                                                                                                        |
| SIG-VLD-0568              | Invalid Operational energy use sub-stage Included.                                                                                               |
| SIG-VLD-0569              | Invalid Operational water use sub-stage Included.                                                                                                |
| SIG-VLD-0570              | Invalid End-of-life stage Included.                                                                                                              |
| SIG-VLD-0571              | Invalid Demolition sub-stage Included.                                                                                                           |
| SIG-VLD-0572              | Invalid Transport sub-stage Included.                                                                                                            |
| SIG-VLD-0573              | Invalid Waste processing sub-stage Included.                                                                                                     |
| SIG-VLD-0574              | Invalid Disposal sub-stage Included.                                                                                                             |
| SIG-VLD-0576              | Distribution stage included must not be null or empty.                                                                                           |
| SIG-VLD-0577              | Usage Stage Included must not be null or empty.                                                                                                  |
| SIG-VLD-0578              | End-of-life stage Included must not be null or empty.                                                                                            |
| SIG-VLD-0579              | Geography Country Subdivision must not be null or empty.                                                                                         |
| SIG-VLD-0580              | PCF Including Biogenic must not be null or empty.                                                                                                |
| SIG-VLD-0581              | Fossil GHG Emissions must not be null or empty.                                                                                                  |
| SIG-VLD-0583              | Biogenic GHG Emissions cannot exceed 15 characters.                                                                                              |
| SIG-VLD-0584              | Biogenic GHG Emissions must be a valid number.                                                                                                   |
| SIG-VLD-0585              | Biogenic GHG Emissions must not be null or empty.                                                                                                |
| SIG-VLD-0586              | dLuc GHG Emissions must not be null or empty.                                                                                                    |
| SIG-VLD-0587              | Aircraft GHG Emissions must not be null or empty.                                                                                                |
| SIG-VLD-0588              | iLuc GHG Emissions cannot exceed 15 characters.                                                                                                  |
| SIG-VLD-0589              | iLuc GHG Emissions must be a valid number.                                                                                                       |
| SIG-VLD-0590              | Distribution stage must not be null or empty.                                                                                                    |
| SIG-VLD-0591              | dLuc GHG Emissions must be a valid number.                                                                                                       |
| SIG-VLD-0592              | luG GHG Emissions cannot exceed 15 characters.                                                                                                   |
| SIG-VLD-0593              | luG GHG Emissions must be a valid number.                                                                                                        |
| SIG-VLD-0594              | Installation stage must not be null or empty.                                                                                                    |
| SIG-VLD-0595              | usage stage must not be null or empty.                                                                                                           |
| SIG-VLD-0596              | eol stage must not be null or empty.                                                                                                             |
| SIG-VLD-0597              | Boundary Processes Description is required.                                                                                                      |
| SIG-VLD-0598              | Other Operator Name is required.                                                                                                                 |
| SIG-VLD-0599              | Characterization Factors Climate Carbon Feedback is required.                                                                                    |
| SIG-VLD-0600              | Uncertainty Assessment Description cannot exceed 2000 characters.                                                                                |
| SIG-VLD-0601              | Invalid Mass Balances Checked.                                                                                                                   |
| SIG-VLD-0602              | Invalid Characterization Factors Climate Carbon Feedback.                                                                                        |
| SIG-VLD-0603              | Invalid Characterization Factors LCIA Method.                                                                                                    |
| SIG-VLD-0604              | Missing network properties. Please update the network properties                                                                                 |
| SIG-VLD-0605              | Network properties `%s` are mandatory to request the pcf data from PACT.                                                                         |
| SIG-VLD-0606              | Invalid PCF standard for PACT. Valid PCF standards for PACT: `%s`                                                                                |
| SIG-VLD-0607              | Bill of material version is mandatory.                                                                                                           |
| SIG-VLD-0608              | Bill of material version length cannot exceed 50 characters.                                                                                     |
| SIG-VLD-0609              | Bill of material comment length cannot exceed 200 characters.                                                                                    |
| SIG-VLD-0610              | Bill of material revision length cannot exceed 200 characters.                                                                                   |
| SIG-VLD-0611              | BOM Version should not contain \<,>,",”,= characters.                                                                                            |
| SIG-VLD-0612              | BOM Comment should not contain \<,>,",”,= characters.                                                                                            |
| SIG-VLD-0613              | BOM Revision should not contain \<,>,",”,= characters.                                                                                           |
| SIG-VLD-0614              | BOM version UUID is invalid.                                                                                                                     |
| SIG-VLD-0615              | Product footprint specification version is mandatory                                                                                             |
| SIG-VLD-0616              | Company name is mandatory                                                                                                                        |
| SIG-VLD-0617              | Product description is mandatory                                                                                                                 |
| SIG-VLD-0618              | Product Classification Code (CPC) is mandatory                                                                                                   |
| SIG-VLD-0619              | Product name is mandatory                                                                                                                        |
| SIG-VLD-0620              | Comment is mandatory                                                                                                                             |
| SIG-VLD-0621              | Unitary Product Amount is mandatory                                                                                                              |
| SIG-VLD-0622              | PCF Excluding Biogenic is mandatory                                                                                                              |
| SIG-VLD-0623              | PCF Including Biogenic is mandatory                                                                                                              |
| SIG-VLD-0624              | Fossil GHG Emissions is mandatory                                                                                                                |
| SIG-VLD-0625              | Fossil Carbon Content is mandatory                                                                                                               |
| SIG-VLD-0626              | Biogenic Carbon Content is mandatory                                                                                                             |
| SIG-VLD-0627              | Boundary Processes Description is mandatory                                                                                                      |
| SIG-VLD-0628              | Exempted Emissions Description is mandatory                                                                                                      |
| SIG-VLD-0629              | Packaging Emissions Included is mandatory                                                                                                        |
| SIG-VLD-0630              | Assurance is mandatory                                                                                                                           |
| SIG-VLD-0631              | Provider Name is mandatory                                                                                                                       |
| SIG-VLD-0632              | Product footprint identifier is mandatory                                                                                                        |
| SIG-VLD-0633              | Status is mandatory and must be any one of \[`%s`\]                                                                                              |
| SIG-VLD-0634              | PCF Data is mandatory                                                                                                                            |
| SIG-VLD-0635              | Created date is mandatory and should be in `yyyy-MM-dd'T'HH:mm:ss'Z'` format.                                                                    |
| SIG-VLD-0636              | Company Id is mandatory                                                                                                                          |
| SIG-VLD-0637              | Product Id is mandatory and should be unique                                                                                                     |
| SIG-VLD-0638              | Reference Period Start date is mandatory and must be in `yyyy-MM-dd'T'HH:mm:ss'Z'`                                                               |
| SIG-VLD-0639              | Reference Period End date is mandatory and must be in `yyyy-MM-dd'T'HH:mm:ss'Z'`                                                                 |
| SIG-VLD-0640              | Declared Unit is mandatory and must be any one of \[`%s`\]                                                                                       |
| SIG-VLD-0641              | Characterization Factors is mandatory and must be any one of \[`%s`\]                                                                            |
| SIG-VLD-0642              | IPCC Characterization Factors Sources is mandatory and must be valid                                                                             |
| SIG-VLD-0643              | Cross Sectoral Standards is mandatory and must match with the list \[`%s`\]                                                                      |
| SIG-VLD-0644              | Exempted Emissions Percent must be between 0.0 and 5.0 (inclusive)                                                                               |
| SIG-VLD-0645              | Technological DQR must be between 1.0 and 3.0 (inclusive)                                                                                        |
| SIG-VLD-0646              | Temporal DQR must be between 1.0 and 3.0 (inclusive)                                                                                             |
| SIG-VLD-0647              | Geographical DQR must be between 1.0 and 3.0 (inclusive)                                                                                         |
| SIG-VLD-0648              | Completeness DQR must be between 1.0 and 3.0 (inclusive)                                                                                         |
| SIG-VLD-0649              | Reliability DQR must be between 1.0 and 3.0 (inclusive)                                                                                          |
| SIG-VLD-0650              | Carbon Content Biogenic is required.                                                                                                             |
| SIG-VLD-0651              | Production stage PCF Including Biogenic must not be null or empty.                                                                               |
| SIG-VLD-0652              | Production stage Fossil GHG Emissions must not be null or empty.                                                                                 |
| SIG-VLD-0653              | Production stage Biogenic GHG Emissions must not be null or empty.                                                                               |
| SIG-VLD-0654              | Production stage dLuc GHG Emissions must not be null or empty.                                                                                   |
| SIG-VLD-0655              | Production stage lu GHG Emissions must not be null or empty.                                                                                     |
| SIG-VLD-0656              | Production stage Aircraft GHG Emissions must not be null or empty.                                                                               |
| SIG-VLD-0657              | Production stage iLuc GHG Emissions must not be null or empty.                                                                                   |
| SIG-VLD-0658              | Production stage PCF Including Biogenic must be a valid number.                                                                                  |
| SIG-VLD-0659              | Production stage Fossil GHG Emissions must be a valid number.                                                                                    |
| SIG-VLD-0660              | Production stage Biogenic GHG Emissions must be a valid number.                                                                                  |
| SIG-VLD-0661              | Production stage dLuc GHG Emissions must be a valid number.                                                                                      |
| SIG-VLD-0662              | Production stage lu GHG Emissions must be a valid number.                                                                                        |
| SIG-VLD-0663              | Production stage Aircraft GHG Emissions must be a valid number.                                                                                  |
| SIG-VLD-0664              | Production stage iLuc GHG Emissions must be a valid number.                                                                                      |
| SIG-VLD-0665              | Production stage PCF Including Biogenic cannot exceed 15 characters.                                                                             |
| SIG-VLD-0666              | Production stage Fossil GHG Emissions cannot exceed 15 characters.                                                                               |
| SIG-VLD-0667              | Production stage Biogenic GHG Emissions cannot exceed 15 characters.                                                                             |
| SIG-VLD-0668              | Production stage dLuc GHG Emissions cannot exceed 15 characters.                                                                                 |
| SIG-VLD-0669              | Production stage lu GHG Emissions cannot exceed 15 characters.                                                                                   |
| SIG-VLD-0670              | Production stage Aircraft GHG Emissions cannot exceed 15 characters.                                                                             |
| SIG-VLD-0671              | Production stage iLuc GHG Emissions cannot exceed 15 characters.                                                                                 |
| SIG-VLD-0672              | Distribution stage PCF Including Biogenic must not be null or empty.                                                                             |
| SIG-VLD-0673              | Distribution stage Fossil GHG Emissions must not be null or empty.                                                                               |
| SIG-VLD-0674              | Distribution stage Biogenic GHG Emissions must not be null or empty.                                                                             |
| SIG-VLD-0675              | Distribution stage dLuc GHG Emissions must not be null or empty.                                                                                 |
| SIG-VLD-0676              | Distribution stage lu GHG Emissions must not be null or empty.                                                                                   |
| SIG-VLD-0677              | Distribution stage Aircraft GHG Emissions must not be null or empty.                                                                             |
| SIG-VLD-0678              | Distribution stage iLuc GHG Emissions must not be null or empty.                                                                                 |
| SIG-VLD-0679              | Distribution stage PCF Including Biogenic must be a valid number.                                                                                |
| SIG-VLD-0680              | Distribution stage Fossil GHG Emissions must be a valid number.                                                                                  |
| SIG-VLD-0681              | Distribution stage Biogenic GHG Emissions must be a valid number.                                                                                |
| SIG-VLD-0682              | Distribution stage dLuc GHG Emissions must be a valid number.                                                                                    |
| SIG-VLD-0683              | Distribution stage lu GHG Emissions must be a valid number.                                                                                      |
| SIG-VLD-0684              | Distribution stage Aircraft GHG Emissions must be a valid number.                                                                                |
| SIG-VLD-0685              | Distribution stage iLuc GHG Emissions must be a valid number.                                                                                    |
| SIG-VLD-0686              | Distribution stage PCF Including Biogenic cannot exceed 15 characters.                                                                           |
| SIG-VLD-0687              | Distribution stage Fossil GHG Emissions cannot exceed 15 characters.                                                                             |
| SIG-VLD-0688              | Distribution stage Biogenic GHG Emissions cannot exceed 15 characters.                                                                           |
| SIG-VLD-0689              | Distribution stage dLuc GHG Emissions cannot exceed 15 characters.                                                                               |
| SIG-VLD-0690              | Distribution stage lu GHG Emissions cannot exceed 15 characters.                                                                                 |
| SIG-VLD-0691              | Distribution stage Aircraft GHG Emissions cannot exceed 15 characters.                                                                           |
| SIG-VLD-0692              | Distribution stage iLuc GHG Emissions cannot exceed 15 characters.                                                                               |
| SIG-VLD-0693              | Installation stage PCF Including Biogenic must not be null or empty.                                                                             |
| SIG-VLD-0694              | Installation stage PCF Including Biogenic must be a valid number.                                                                                |
| SIG-VLD-0695              | Installation stage PCF Including Biogenic cannot exceed 15 characters.                                                                           |
| SIG-VLD-0696              | Usage stage PCF Including Biogenic must not be null or empty.                                                                                    |
| SIG-VLD-0697              | Usage stage PCF Including Biogenic must be a valid number.                                                                                       |
| SIG-VLD-0698              | Usage stage PCF Including Biogenic cannot exceed 15 characters.                                                                                  |
| SIG-VLD-0699              | End-of-life stage PCF Including Biogenic must not be null or empty.                                                                              |
| SIG-VLD-0700              | End-of-life stage PCF Including Biogenic must be a valid number.                                                                                 |
| SIG-VLD-0701              | End-of-life stage PCF Including Biogenic cannot exceed 15 characters.                                                                            |
| SIG-VLD-0702              | Characterization Factors Time Horizon must not be null or empty.                                                                                 |
| SIG-VLD-0703              | Duplicate company identifier types present in the request.                                                                                       |
| SIG-VLD-0704              | No company identifiers found in the request.                                                                                                     |
| SIG-VLD-0705              | Eol demolition sub stage PCF Including Biogenic must be a valid number.                                                                          |
| SIG-VLD-0706              | Eol demolition sub stage PCF Including Biogenic cannot exceed 15 characters.                                                                     |
| SIG-VLD-0707              | Eol transport sub stage PCF Including Biogenic must be a valid number.                                                                           |
| SIG-VLD-0708              | Eol transport sub stage PCF Including Biogenic cannot exceed 15 characters.                                                                      |
| SIG-VLD-0709              | Eol waster processing sub stage PCF Including Biogenic must be a valid number.                                                                   |
| SIG-VLD-0710              | Eol waster processing sub stage PCF Including Biogenic cannot exceed 15 characters.                                                              |
| SIG-VLD-0711              | Eol disposal sub stage PCF Including Biogenic must be a valid number.                                                                            |
| SIG-VLD-0712              | Eol disposal sub stage PCF Including Biogenic cannot exceed 15 characters.                                                                       |
| SIG-VLD-0713              | Usage maintenance sub stage PCF Including Biogenic must be a valid number.                                                                       |
| SIG-VLD-0714              | Usage maintenance sub stage PCF Including Biogenic cannot exceed 15 characters.                                                                  |
| SIG-VLD-0715              | Usage repair sub stage PCF Including Biogenic must be a valid number.                                                                            |
| SIG-VLD-0716              | Usage repair sub stage PCF Including Biogenic cannot exceed 15 characters.                                                                       |
| SIG-VLD-0717              | Usage use sub stage PCF Including Biogenic must be a valid number.                                                                               |
| SIG-VLD-0718              | Usage use sub stage PCF Including Biogenic cannot exceed 15 characters.                                                                          |
| SIG-VLD-0719              | Usage replacement sub stage PCF Including Biogenic must be a valid number.                                                                       |
| SIG-VLD-0720              | Usage replacement sub stage PCF Including Biogenic cannot exceed 15 characters.                                                                  |
| SIG-VLD-0721              | Usage eol treatment of generated waste use sub stage PCF Including Biogenic must be a valid number.                                              |
| SIG-VLD-0722              | Usage eol treatment of generated waste use sub stage PCF Including Biogenic cannot exceed 15 characters.                                         |
| SIG-VLD-0723              | Usage refurbishment sub stage PCF Including Biogenic must be a valid number.                                                                     |
| SIG-VLD-0724              | Usage refurbishment sub stage PCF Including Biogenic cannot exceed 15 characters.                                                                |
| SIG-VLD-0725              | Usage operational energy use sub stage PCF Including Biogenic must be a valid number.                                                            |
| SIG-VLD-0726              | Usage operational energy use sub stage PCF Including Biogenic cannot exceed 15 characters.                                                       |
| SIG-VLD-0727              | Usage operational water use sub stage PCF Including Biogenic must be a valid number.                                                             |
| SIG-VLD-0728              | Usage operational water use sub stage PCF Including Biogenic cannot exceed 15 characters.                                                        |
| SIG-VLD-0729              | Installation sub stage PCF Including Biogenic must be a valid number.                                                                            |
| SIG-VLD-0730              | Installation sub stage PCF Including Biogenic cannot exceed 15 characters.                                                                       |
| SIG-VLD-0731              | Installation Eol treatment of generated waste installation sub stage PCF Including Biogenic must be a valid number.                              |
| SIG-VLD-0732              | Installation Eol treatment of generated waste installation sub stage PCF Including Biogenic cannot exceed 15 characters.                         |
| SIG-VLD-0733              | Production material acquisition and pre processing raw material acquisition sub stage PCF Including Biogenic must be a valid number.             |
| SIG-VLD-0734              | Production material acquisition and pre processing raw material acquisition sub stage PCF Including Biogenic cannot exceed 15 characters.        |
| SIG-VLD-0735              | Production material acquisition and pre processing transport to manufacturing site sub stage PCF Including Biogenic must be a valid number.      |
| SIG-VLD-0736              | Production material acquisition and pre processing transport to manufacturing site sub stage PCF Including Biogenic cannot exceed 15 characters. |
| SIG-VLD-0737              | Production material acquisition and pre processing pre processing sub stage PCF Including Biogenic must be a valid number.                       |
| SIG-VLD-0738              | Production material acquisition and pre processing pre processing sub stage PCF Including Biogenic cannot exceed 15 characters.                  |
| SIG-VLD-0739              | Production manufacturing packaging sub stage PCF Including Biogenic must be a valid number.                                                      |
| SIG-VLD-0740              | Production manufacturing packaging sub stage PCF Including Biogenic cannot exceed 15 characters.                                                 |
| SIG-VLD-0741              | Production manufacturing assembly sub stage PCF Including Biogenic must be a valid number.                                                       |
| SIG-VLD-0742              | Production manufacturing assembly sub stage PCF Including Biogenic cannot exceed 15 characters.                                                  |
| SIG-VLD-0743              | Production manufacturing Eol treatment of generated waste installation sub stage PCF Including Biogenic must be a valid number.                  |
| SIG-VLD-0744              | Production manufacturing Eol treatment of generated waste installation sub stage PCF Including Biogenic cannot exceed 15 characters.             |
| SIG-VLD-0745              | Distribution transport A to distributor sub stage PCF Including Biogenic must be a valid number.                                                 |
| SIG-VLD-0746              | Distribution transport A to distributor sub stage PCF Including Biogenic cannot exceed 15 characters.                                            |
| SIG-VLD-0747              | Distribution reconditioning A to distributor sub stage PCF Including Biogenic must be a valid number.                                            |
| SIG-VLD-0748              | Distribution reconditioning A to distributor sub stage PCF Including Biogenic cannot exceed 15 characters.                                       |
| SIG-VLD-0749              | Distribution transport B to place of use sub stage PCF Including Biogenic must be a valid number.                                                |
| SIG-VLD-0750              | Distribution transport B to place of use sub stage PCF Including Biogenic cannot exceed 15 characters.                                           |
| SIG-VLD-0751              | Production manufacturing sub stage PCF Including Biogenic must be a valid number.                                                                |
| SIG-VLD-0752              | Production manufacturing sub stage PCF Including Biogenic cannot exceed 15 characters.                                                           |
| SIG-VLD-0753              | Production material acquisition and pre processing sub stage PCF Including Biogenic must be a valid number.                                      |
| SIG-VLD-0754              | Production material acquisition and pre processing sub stage PCF Including Biogenic cannot exceed 15 characters.                                 |
| SIG-VLD-0755              | String.format("Invalid request event type and it must be any of \[`%s`\]",PactRequestTypeEnum.getSupportedRequestTypes())                        |
| SIG-VLD-0756              | specVersion must be more than or equal to 5 characters.                                                                                          |
| SIG-VLD-0757              | specVersion is invalid.                                                                                                                          |
| SIG-VLD-0758              | Event id must not be empty                                                                                                                       |
| SIG-VLD-0759              | source of the request must be less than or equal to 200 characters.                                                                              |
| SIG-VLD-0760              | source of the request must be more than or equal to 1 character.                                                                                 |
| SIG-VLD-0761              | requestEventId must be less than or equal to 36 characters.                                                                                      |
| SIG-VLD-0762              | Pf id must be less than or equal to 36 characters.                                                                                               |
| SIG-VLD-0763              | Pf specVersion must be more than or equal to 5 characters.                                                                                       |
| SIG-VLD-0764              | Pf specVersion is invalid.                                                                                                                       |
| SIG-VLD-0765              | Invalid PACT Pf status.                                                                                                                          |
| SIG-VLD-0766              | status comment must be less than or equal to 200 characters.                                                                                     |
| SIG-VLD-0767              | Pf production description must be less than or equal to 200 characters.                                                                          |
| SIG-VLD-0768              | Pf product category cpc must be less than or equal to 200 characters.                                                                            |
| SIG-VLD-0769              | Pf product category cpc must be more than or equal to 1 character.                                                                               |
| SIG-VLD-0770              | Pf product name company must be less than or equal to 200 characters.                                                                            |
| SIG-VLD-0771              | Pf comment must be less than or equal to 200 characters.                                                                                         |
| SIG-VLD-0772              | Invalid unitary product amount                                                                                                                   |
| SIG-VLD-0773              | Invalid pcfExcludingBiogenic                                                                                                                     |
| SIG-VLD-0774              | Invalid pcfIncludingBiogenic                                                                                                                     |
| SIG-VLD-0775              | Invalid characterizationFactors                                                                                                                  |
| SIG-VLD-0776              | Invalid declared unit                                                                                                                            |
| SIG-VLD-0777              | ipccCharacterizationFactorsSources must contain at least 1 value                                                                                 |
| SIG-VLD-0778              | ipccCharacterizationFactorsSources must contain unique values                                                                                    |
| SIG-VLD-0779              | ipccCharacterizationFactorsSources values are invalid                                                                                            |
| SIG-VLD-0780              | crossSectoralStandardsUsed is invalid                                                                                                            |
| SIG-VLD-0781              | Invalid product or sector specific rule operator                                                                                                 |
| SIG-VLD-0782              | ruleNames must contain at least 1 element                                                                                                        |
| SIG-VLD-0783              | ruleNames must contain unique values                                                                                                             |
| SIG-VLD-0784              | each rule should have character length at least 1                                                                                                |
| SIG-VLD-0785              | each rule should have character length not more than 100                                                                                         |
| SIG-VLD-0786              | Invalid biogenic accounting methodology                                                                                                          |
| SIG-VLD-0787              | Boundary Processes Description length should not be more than 300                                                                                |
| SIG-VLD-0788              | SecondaryEmissionFactorSources version should be at least 1 character                                                                            |
| SIG-VLD-0789              | geographyCountrySubdivision length should be at least 1 character                                                                                |
| SIG-VLD-0790              | geographyCountry length should not be more than 2 characters                                                                                     |
| SIG-VLD-0791              | geographyCountry length should be at least 1 character                                                                                           |
| SIG-VLD-0792              | Invalid geographyRegionOrSubregion                                                                                                               |
| SIG-VLD-0793              | SecondaryEmissionFactorSources name length should not be more than 100 characters                                                                |
| SIG-VLD-0794              | SecondaryEmissionFactorSources name should be at least 1 character                                                                               |
| SIG-VLD-0795              | exemptedEmissionsDescription length should not be more than 200 characters                                                                       |
| SIG-VLD-0796              | Invalid packagingGhgEmissions                                                                                                                    |
| SIG-VLD-0797              | Invalid assurance coverage                                                                                                                       |
| SIG-VLD-0798              | Invalid assurance level                                                                                                                          |
| SIG-VLD-0799              | Invalid assurance boundary                                                                                                                       |
| SIG-VLD-0800              | Invalid PFErrorCode                                                                                                                              |
| SIG-VLD-0801              | geographyCountrySubdivision length should not be more than 50 characters                                                                         |
| SIG-VLD-0802              | SecondaryEmissionFactorSources version length should not be more than 100 characters                                                             |
| SIG-VLD-0803              | Comment must be less than or equal to 3000 characters.                                                                                           |
| SIG-VLD-0804              | Group identifier is mandatory.                                                                                                                   |
| SIG-VLD-0805              | Quantity is mandatory.                                                                                                                           |
| SIG-VLD-0806              | Group name is mandatory.                                                                                                                         |
| SIG-VLD-0807              | Component identifier is mandatory.                                                                                                               |
| SIG-VLD-0808              | Product identifier is mandatory.                                                                                                                 |
| SIG-VLD-0809              | Quantity value cannot exceed 15 digits, including the decimal point.                                                                             |
| SIG-VLD-0810              | Group identifier length should not be more than 100 characters.                                                                                  |
| SIG-VLD-0811              | Group name length should not be more than 100 characters.                                                                                        |
| SIG-VLD-0812              | Component identifier length should not be more than 100 characters.                                                                              |
| SIG-VLD-0813              | Component id length should not be more than 36 characters.                                                                                       |
| SIG-VLD-0814              | Product identifier length should not be more than 100 characters.                                                                                |
| SIG-VLD-0815              | Product id length should not be more than 36 characters.                                                                                         |
| SIG-VLD-0816              | BOM version Id (UUID) must not be null or empty                                                                                                  |
| SIG-VLD-0817              | BOM item Id (UUID) must not be null or empty                                                                                                     |
| SIG-VLD-0818              | Name should not exceed 50 characters                                                                                                             |
| SIG-VLD-0819              | Description should not exceed 250 characters                                                                                                     |
| SIG-VLD-0820              | ClientId should not exceed 50 characters                                                                                                         |
| SIG-VLD-0821              | ClientSecret should not exceed 250 characters                                                                                                    |
| SIG-VLD-0822              | scopes should not exceed 100 characters                                                                                                          |
| SIG-VLD-0823              | Token URL should not exceed 100 characters                                                                                                       |
| SIG-VLD-0824              | Base URL should not exceed 100 characters                                                                                                        |
| SIG-VLD-0825              | Audience should not exceed 250 characters                                                                                                        |
| SIG-VLD-0826              | Last connection time should not exceed 100 characters                                                                                            |
| SIG-VLD-0827              | Configuration name already exists for company                                                                                                    |
| SIG-VLD-0828              | Given configuration id does not exists for company                                                                                               |
| SIG-VLD-0829              | PACT network not exists                                                                                                                          |
| SIG-VLD-0830              | Failed to create technical user                                                                                                                  |
| SIG-VLD-0831              | Cannot add parent product as BOM sub item.                                                                                                       |
| SIG-VLD-0832              | BOM sub-items cannot be empty.                                                                                                                   |
| SIG-VLD-0833              | BOM item `%s` cannot be repeated inside same level.                                                                                              |
| SIG-VLD-0834              | BOM item `%s` cannot be repeated inside same level.                                                                                              |
| SIG-VLD-0835              | Partner company name already exists for company.                                                                                                 |
| SIG-VLD-0836              | Partner clientId is required.                                                                                                                    |
| SIG-VLD-0837              | Partner clientSecret is required.                                                                                                                |
| SIG-VLD-0838              | Partner tokenUrl is required.                                                                                                                    |
| SIG-VLD-0839              | Partner baseUrl is required.                                                                                                                     |
| SIG-VLD-0840              | Your company is not enabled for PACT feature.                                                                                                    |
| SIG-VLD-0841              | Invalid Last connection tested on Format. Please send valid date format (for eg: "")                                                             |
| SIG-VLD-0842              | No pcf data found                                                                                                                                |
| SIG-VLD-0843              | The version of the pcf with value an integer in the inclusive range of 0 to 2^32-1                                                               |
| SIG-VLD-0844              | Unitary Product Amount must be a greater than zero                                                                                               |
| SIG-VLD-0845              | Invalid product including biogenic CO2 emission                                                                                                  |
| SIG-VLD-0846              | Invalid product excluding biogenic CO2 emission value and must be greater than or equal to 0                                                     |
| SIG-VLD-0847              | Invalid fossil ghg emission value and must be greater than or equal to 0                                                                         |
| SIG-VLD-0848              | Invalid fossil carbon content CO2 emission value and must be greater than or equal to 0                                                          |
| SIG-VLD-0849              | Invalid biogenic carbon content CO2 emission value and must be greater than or equal to 0                                                        |
| SIG-VLD-0850              | Invalid dLuc GHG CO2 emission value and must be greater than or equal to 0                                                                       |
| SIG-VLD-0851              | Invalid lu GHG emissions                                                                                                                         |
| SIG-VLD-0852              | Invalid aircraft GHG Emissions value and must be greater than or equal to 0                                                                      |
| SIG-VLD-0853              | Invalid other biogenic GHG emission value and must be greater than or equal to 0                                                                 |
| SIG-VLD-0854              | Invalid iLuc GHG emission value and must be greater than or equal to 0                                                                           |
| SIG-VLD-0855              | Invalid Biogenic Carbon Withdrawal                                                                                                               |
| SIG-VLD-0856              | Invalid Biogenic Accounting Methodology and must match with \[`%s`\]                                                                             |
| SIG-VLD-0857              | Packaging Ghg emission value must be greater than or equal to 0                                                                                  |
| SIG-VLD-0858              | Given PackagingEmissionsIncluded is false, but PackagingGhgEmissions is provided                                                                 |
| SIG-VLD-0859              | Partner configuration Id (UUID) must not be null or empty                                                                                        |
| SIG-VLD-0860              | Partner configuration Id (UUID) must be less than or equal to 36 characters.                                                                     |
| SIG-VLD-0861              | Partner Token url should start with https://.                                                                                                    |
| SIG-VLD-0862              | Partner Base url should start with https://.                                                                                                     |
| SIG-VLD-0863              | Invalid Biogenic Accounting Methodology.                                                                                                         |
| SIG-VLD-0864              | Version is required.                                                                                                                             |
| SIG-VLD-0865              | Version cannot exceed 15 characters.                                                                                                             |
| SIG-VLD-0866              | Coverage percent cannot exceed 15 characters.                                                                                                    |
| SIG-VLD-0867              | Uncertainty Assessment Description cannot exceed 1000 characters.                                                                                |
| SIG-VLD-0871              | Partner configuration name is required.                                                                                                          |
| SIG-VLD-0872              | Fossil Carbon Content cannot exceed 15 characters.                                                                                               |
| SIG-VLD-0873              | Necessary credentials PACT partner for sharing PCF is not available                                                                              |
| SIG-VLD-0874              | Partner configuration is required.                                                                                                               |
| SIG-VLD-0875              | Company name should not exceed 50 characters.                                                                                                    |
| SIG-VLD-0876              | Company name is required.                                                                                                                        |
| SIG-VLD-0877              | Configuration name cannot contain \<,>,",”,= characters.                                                                                         |
| SIG-VLD-0878              | Suggestion Request cannot be null                                                                                                                |
| SIG-VLD-0879              | Cannot specify both source field and exclude_source fields.                                                                                      |
| SIG-VLD-0880              | Use valid source name for source/exclude_source fields                                                                                           |
| SIG-VLD-0881              | Company name cannot contain \<,>,",”,= characters.                                                                                               |
| SIG-VLD-0882              | Audience should start with https://.                                                                                                             |
| SIG-VLD-0883              | Client id, client secret, token url and base url all are required                                                                                |
| SIG-VLD-0884              | Source is not valid for given domain                                                                                                             |
| SIG-VLD-0885              | Partner configuration not exist for company                                                                                                      |
| SIG-VLD-0890              | Invalid date format of `%s`. Please provide date in yyyy-MM-dd'T'HHss'Z' format.                                                                 |
| SIG-VLD-0891              | Fossil Carbon Content must be a valid number.                                                                                                    |
| SIG-VLD-0892              | updated must be a valid date(Eg: 2024-02-01T08:30:00.000Z).                                                                                      |
| SIG-VLD-0893              | coveragePercent must be a number between 0 and 100.                                                                                              |
| SIG-VLD-0894              | productCodeCpc is mandatory                                                                                                                      |
| SIG-VLD-0895              | Invalid product footprint specification version                                                                                                  |
| SIG-VLD-0896              | Invalid request event Id                                                                                                                         |
| SIG-VLD-0897              | Unable to process packaging json                                                                                                                 |
| SIG-VLD-0898              | Invalid unit types passed, valid values are Weight,Volume,Number                                                                                 |
| SIG-VLD-0899              | Client ID is mandatory for requesting PCF via PACT network. Please check PACT configuration.                                                     |
| SIG-VLD-0900              | Invalid date format of `%s`. Please provide date in yyyy-MM-dd'T'HHss.SSS'Z' format.                                                             |
| SIG-VLD-0901              | Invalid ProductOrSectorSpecificRule Operator                                                                                                     |
| SIG-VLD-0902              | Missing ProductOrSectorSpecificRule RuleName                                                                                                     |
| SIG-VLD-0903              | Missing ProductOrSectorSpecificRule OtherOperatorName                                                                                            |
| SIG-VLD-0904              | ProductOrSectorSpecificRule Operator is not Other, but otherOperatorName is provided in request                                                  |
| SIG-VLD-0905              | Please provide mass as PACT v2 connector does not allow to send pieces                                                                           |
| SIG-VLD-0906              | UnitaryProductAmount value must be greater than 0                                                                                                |
| SIG-VLD-0907              | Invalid BiogenicCarbonWithdrawal value and must be less than or equal to 0                                                                       |
| SIG-VLD-0908              | Invalid geographyRegionOrSubregion value must contains `%s`                                                                                      |
| SIG-VLD-0909              | PackagingGhgEmissions emission value must be greater than or equal to 0                                                                          |
| SIG-VLD-0910              | No productIds found in the request                                                                                                               |
| SIG-VLD-0911              | Level must not be empty and must be a positive number greater than zero.                                                                         |
| SIG-VLD-0912              | Internal product must be true or false.                                                                                                          |
| SIG-VLD-0913              | No valid level found for this item.                                                                                                              |
| SIG-VLD-0914              | Nested level limit exceeded. Only `%s` level allowed.                                                                                            |
| SIG-VLD-0915              | Deprecated PCFs are not accepted. Please send an updated PCF.                                                                                    |
| SIG-VLD-0916              | BOM Item UUID must be less than or equal to 36 characters.                                                                                       |
| SIG-VLD-0917              | Max uploaded image size is limited to `%s`                                                                                                       |
| SIG-VLD-0918              | Contact number is limited to 30 characters.                                                                                                      |
| SIG-VLD-0919              | Address is limited to 1000 characters.                                                                                                           |
| SIG-VLD-0920              | Address cannot contain \<,>,",”,= characters.                                                                                                    |
| SIG-VLD-0921              | Invalid include distribution stage.                                                                                                              |
| SIG-VLD-0922              | Invalid production stage.                                                                                                                        |
| SIG-VLD-0923              | Country is limited to 50 characters.                                                                                                             |
| SIG-VLD-0924              | Country is mandatory.                                                                                                                            |
| SIG-VLD-0925              | Country cannot contain \<,>,(,),",”,= characters.                                                                                                |
| SIG-VLD-0926              | ZipCode is limited to 10 digits.                                                                                                                 |
| SIG-VLD-0927              | ZipCode cannot contain \<,>,",”,= characters.                                                                                                    |
| SIG-VLD-0928              | Industry is limited to 50 characters.                                                                                                            |
| SIG-VLD-0929              | Website is limited to 200 characters.                                                                                                            |
| SIG-VLD-0930              | Description is limited to 1000 characters.                                                                                                       |
| SIG-VLD-0931              | Industry is mandatory.                                                                                                                           |
| SIG-VLD-0932              | Website is mandatory.                                                                                                                            |
| SIG-VLD-0933              | Invalid assessment year.                                                                                                                         |
| SIG-VLD-0935              | onboarding completed is a mandatory field.                                                                                                       |
| SIG-VLD-0936              | Allocation Rules Description cannot exceed 5000 characters.                                                                                      |
| SIG-VLD-0937              | Secondary Emission Factor Sources cannot exceed 1000 characters.                                                                                 |
| SIG-VLD-0938              | Columns are mandatory                                                                                                                            |
| SIG-VLD-0939              | Invalid filter column                                                                                                                            |
| SIG-VLD-0940              | Invalid supplier column                                                                                                                          |
| SIG-VLD-0941              | sort key `%s` not found                                                                                                                          |
| SIG-VLD-0942              | Partner SiGREEN id can not be blank.                                                                                                             |
| SIG-VLD-0943              | Message to partner must be less than 250 characters                                                                                              |
| SIG-VLD-0944              | Message to partner can not be blank.                                                                                                             |
| SIG-VLD-0945              | Partner Request (UUID) must be less than or equal to 36 characters.                                                                              |
| SIG-VLD-0946              | Partner Request (UUID) must not be null or empty                                                                                                 |
| SIG-VLD-0947              | Invalid Partner Request (UUID).                                                                                                                  |
| SIG-VLD-0948              | Partner SiGREEN id must be less than or equal to 14 characters.                                                                                  |
| SIG-VLD-0949              | Partner request does not exist.                                                                                                                  |
| SIG-VLD-0950              | PCF ID is not part of the selected product.                                                                                                      |
| SIG-VLD-0951              | Include distribution stage must be true or false.                                                                                                |
| SIG-VLD-0952              | Product PCF ID only allowed for product item.                                                                                                    |
| SIG-VLD-0953              | Include distribution stage only allow for product item.                                                                                          |
| SIG-VLD-0954              | Invalid SiGREEN id.                                                                                                                              |
| SIG-VLD-0955              | A company cannot send a request to itself.                                                                                                       |
| SIG-VLD-0956              | Partner company cannot cancel the request.                                                                                                       |
| SIG-VLD-0957              | A company cannot accept its own request.                                                                                                         |
| SIG-VLD-0958              | A company cannot reject its own request.                                                                                                         |
| SIG-VLD-0959              | Search text must be between 1 and 50 characters.                                                                                                 |
| SIG-VLD-0960              | Partner Message (UUID) must be less than or equal to 36 characters.                                                                              |
| SIG-VLD-0961              | Invalid Partner Message (UUID).                                                                                                                  |
| SIG-VLD-0962              | No Partner Message (UUID) provided.                                                                                                              |
| SIG-VLD-0963              | A company cannot send a message to itself.                                                                                                       |
| SIG-VLD-0964              | Invalid Partner Network Message Access.                                                                                                          |
| SIG-VLD-0965              | Invalid quantity specified `%s`.                                                                                                                 |
| SIG-VLD-0966              | Task Comment not found for task UID `%s` and comment UID `%s`.                                                                                   |
| SIG-VLD-0967              | Invalid search text. It should not contain =, \<, >, ' and " characters.                                                                         |
| SIG-VLD-0968              | Invalid industry                                                                                                                                 |
| SIG-VLD-0969              | Invalid country                                                                                                                                  |
| SIG-VLD-0970              | Maximum `%s` BOM items can be uploaded in a single request.                                                                                      |
| SIG-VLD-0971              | Invalid partner network connection request status type                                                                                           |
| SIG-VLD-0972              | Partner network connection request status type is required                                                                                       |
| SIG-VLD-0973              | Partner Requester (UUID) must not be null or empty                                                                                               |
| SIG-VLD-0974              | Partner Requester (UUID) must not be null or empty                                                                                               |
| SIG-VLD-0975              | Invalid search text.                                                                                                                             |
| SIG-VLD-0976              | Factory emission data not found.                                                                                                                 |
| SIG-VLD-0977              | Invalid GeographySubRegion.                                                                                                                      |
| SIG-VLD-0978              | Invalid Upstream Emission data (UUID).                                                                                                           |
| SIG-VLD-0979              | Product family name cannot be null or empty.                                                                                                     |
| SIG-VLD-0980              | Product family name already exists in company.                                                                                                   |
| SIG-VLD-0981              | Search text must be between 1 and 100 characters.                                                                                                |
| SIG-VLD-0982              | Bulk deletion is limited to a maximum of 100 BOM items per request.                                                                              |
| SIG-VLD-0983              | Invalid Characterization Factors TimeHorizon.                                                                                                    |
| SIG-VLD-0984              | PCF not found                                                                                                                                    |
| SIG-VLD-0985              | Proof Id (UUID) must be less than or equal to 36 characters.                                                                                     |
| SIG-VLD-0986              | Proof Id (UUID) must not be null or empty                                                                                                        |
| SIG-VLD-0987              | Invalid Proof Id (UUID).                                                                                                                         |
| SIG-VLD-0988              | Bom Group identifier cannot contain " " characters.                                                                                              |
| SIG-VLD-0989              | Bom Group name cannot contain " " characters.                                                                                                    |
| SIG-VLD-0990              | Comment cannot contain " " characters.                                                                                                           |
| SIG-VLD-0991              | SourceSystem cannot contain " " characters.                                                                                                      |
| SIG-VLD-0992              | Product description cannot contain " " characters.                                                                                               |
| SIG-VLD-0993              | Batch number cannot contain " " characters.                                                                                                      |
| SIG-VLD-0994              | Data source cannot contain " " characters.                                                                                                       |
| SIG-VLD-0995              | Batch comment cannot contain " " characters.                                                                                                     |
| SIG-VLD-0996              | Type of activity cannot contain " " characters.                                                                                                  |
| SIG-VLD-0997              | Emission activity comment cannot contain " " characters.                                                                                         |
| SIG-VLD-0998              | Reject reason cannot contain " " characters.                                                                                                     |
| SIG-VLD-0999              | Task Comment cannot contain " " characters.                                                                                                      |
| SIG-VLD-1000              | Task Comment must not be null or empty.                                                                                                          |
| SIG-VLD-1001              | Task Comment cannot exceed 1000 characters.                                                                                                      |
| SIG-VLD-1002              | Legal statement cannot contain " " characters.                                                                                                   |
| SIG-VLD-1003              | Comment cannot contain " " characters.                                                                                                           |
| SIG-VLD-1004              | Purchase Order Details cannot contain " " characters.                                                                                            |
| SIG-VLD-1005              | Component Supplier is deleted, cannot accept or reject proof.                                                                                    |
| SIG-VLD-1006              | Emission item name cannot be blank.                                                                                                              |
| SIG-VLD-1007              | Emission item length should not be more than 100 characters.                                                                                     |
| SIG-VLD-1008              | Emission item identifier length should not be more than 100 characters.                                                                          |
| SIG-VLD-1009              | Emission item unitType is invalid.                                                                                                               |
| SIG-VLD-1010              | Invalid column `%s`                                                                                                                              |
| SIG-VLD-1011              | Invalid search column `%s`                                                                                                                       |
| SIG-VLD-1012              | Task Comment Id (UUID) must not be null or empty                                                                                                 |
| SIG-VLD-1013              | Invalid Task Comment Id (UUID).                                                                                                                  |

## Business Errors

| **Validation Error Code** | **Error Description**                                                                                                                                                                                                    |
| ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| SIG-BUS-0001              | The Factory Name for is already in use. Please choose a different name.                                                                                                                                                  |
| SIG-BUS-0002              | The Factory Id is invalid. Please verify and provide a valid Factory Id.                                                                                                                                                 |
| SIG-BUS-0003              | The deletion of the Factory is not possible as it is linked to products.                                                                                                                                                 |
| SIG-BUS-0004              | Duplicate Factory not allowed in company with same name.                                                                                                                                                                 |
| SIG-BUS-0005              | Company not found.                                                                                                                                                                                                       |
| SIG-BUS-0006              | The company already has a product with the identical identifier.                                                                                                                                                         |
| SIG-BUS-0007              | The identifier type `%s` is invalid. Please verify and provide a valid identifier type.                                                                                                                                  |
| SIG-BUS-0008              | The main identifier type is invalid. Please verify and provide a valid main identifier type.                                                                                                                             |
| SIG-BUS-0009              | The company has not yet defined any identifier types.                                                                                                                                                                    |
| SIG-BUS-0010              | Provided unit type `%s` is not supported.                                                                                                                                                                                |
| SIG-BUS-0011              | Weight is invalid.                                                                                                                                                                                                       |
| SIG-BUS-0012              | Weight should be empty for unit type 'kg'.                                                                                                                                                                               |
| SIG-BUS-0013              | Access denied.                                                                                                                                                                                                           |
| SIG-BUS-0014              | Product family `%s` does not exists.                                                                                                                                                                                     |
| SIG-BUS-0015              | Factory does not exists `%s`.                                                                                                                                                                                            |
| SIG-BUS-0016              | Product does not exists.                                                                                                                                                                                                 |
| SIG-BUS-0017              | The main identifier value of the product cannot be altered.                                                                                                                                                              |
| SIG-BUS-0018              | Invalid factory name `%s`.                                                                                                                                                                                               |
| SIG-BUS-0019              | Mismatch in identifier default value, review `%s`.                                                                                                                                                                       |
| SIG-BUS-0020              | No credential found for requested product.                                                                                                                                                                               |
| SIG-BUS-0021              | The identifier list does not permit duplicate identifier types.                                                                                                                                                          |
| SIG-BUS-0022              | The name for the identifier type is already in use. Please choose a different name.                                                                                                                                      |
| SIG-BUS-0023              | The provided Identifier type id does not exist.                                                                                                                                                                          |
| SIG-BUS-0024              | Main Identifier type cannot be deleted.                                                                                                                                                                                  |
| SIG-BUS-0025              | The deletion of the product identifier type is not possible as it is linked to products.                                                                                                                                 |
| SIG-BUS-0026              | Your account has reached the maximum allowed number of identifiers, which is `%s`. Additional identifiers cannot be added.                                                                                               |
| SIG-BUS-0027              | The deletion of the Factory is not possible as it is linked to products.                                                                                                                                                 |
| SIG-BUS-0028              | Main identifier value cannot be modified.                                                                                                                                                                                |
| SIG-BUS-0029              | Component Id (UUID) is not associated with identifier.                                                                                                                                                                   |
| SIG-BUS-0030              | Supplier not exist for given supplier identifier.                                                                                                                                                                        |
| SIG-BUS-0031              | Component already exists with the same identifier in the company.                                                                                                                                                        |
| SIG-BUS-0032              | Component does not exist.                                                                                                                                                                                                |
| SIG-BUS-0033              | Supplier does not exist.                                                                                                                                                                                                 |
| SIG-BUS-0034              | Supplier location does not exist.                                                                                                                                                                                        |
| SIG-BUS-0035              | Secondary data does not exist.                                                                                                                                                                                           |
| SIG-BUS-0036              | Secondary data is not associated with component.                                                                                                                                                                         |
| SIG-BUS-0037              | Supplier Location already exists for component supplier combination.                                                                                                                                                     |
| SIG-BUS-0038              | Supplier Product Id already exists for component supplier combination.                                                                                                                                                   |
| SIG-BUS-0039              | Supplier Location cannot be modified, active proof is present.                                                                                                                                                           |
| SIG-BUS-0040              | Supplier Location cannot be modified, secondary data is present.                                                                                                                                                         |
| SIG-BUS-0041              | Unit cannot be modified.                                                                                                                                                                                                 |
| SIG-BUS-0042              | Supplier already exists with the same identifier in the company.                                                                                                                                                         |
| SIG-BUS-0043              | Component Identifier Type cannot be deleted because it is linked to components.                                                                                                                                          |
| SIG-BUS-0044              | Supplier does not exist.                                                                                                                                                                                                 |
| SIG-BUS-0045              | Share should be within 0 to 100                                                                                                                                                                                          |
| SIG-BUS-0046              | Supplier Location should be unique for component supplier combination.                                                                                                                                                   |
| SIG-BUS-0047              | Supplier Product Id should be unique for component supplier combination.                                                                                                                                                 |
| SIG-BUS-0048              | Component Id (UUID) does not exist                                                                                                                                                                                       |
| SIG-BUS-0049              | Secondary Data Id (UUID) does not exist                                                                                                                                                                                  |
| SIG-BUS-0050              | Supplier Identifier Type cannot be deleted, because it is linked to suppliers.                                                                                                                                           |
| SIG-BUS-0051              | Component Supplier link does not exist.                                                                                                                                                                                  |
| SIG-BUS-0052              | Component Supplier link not associated with given Component Id (UUID).                                                                                                                                                   |
| SIG-BUS-0054              | Incorrect Component Ids found in the request. Please send valid Component Ids                                                                                                                                            |
| SIG-BUS-0055              | Pcf Id does not exist.                                                                                                                                                                                                   |
| SIG-BUS-0056              | Error Response from `%s` for getting `%s`                                                                                                                                                                                |
| SIG-BUS-0057              | Error when trying to connect `%s` for getting `%s`                                                                                                                                                                       |
| SIG-BUS-0058              | Error while parsing `%s`                                                                                                                                                                                                 |
| SIG-BUS-0059              | Error while parsing from and to date to UTC format                                                                                                                                                                       |
| SIG-BUS-0060              | Unable to send notification.                                                                                                                                                                                             |
| SIG-BUS-0061              | Error while parsing the activity message.                                                                                                                                                                                |
| SIG-BUS-0062              | Material category image does not exist.                                                                                                                                                                                  |
| SIG-BUS-0063              | Category with same name already exists                                                                                                                                                                                   |
| SIG-BUS-0064              | Category with name Uncategorized is not allowed                                                                                                                                                                          |
| SIG-BUS-0065              | Material category Id does not exist.                                                                                                                                                                                     |
| SIG-BUS-0066              | File scan not completed                                                                                                                                                                                                  |
| SIG-BUS-0067              | Error occurred while file upload and scan                                                                                                                                                                                |
| SIG-BUS-0068              | Material Category image file is infected.                                                                                                                                                                                |
| SIG-BUS-0069              | Factory doesn't exist in the company.                                                                                                                                                                                    |
| SIG-BUS-0070              | Factory-product mapping not found. If you would like to add these factory emissions for this product, please ensure that the product is associated with the factory. This is done via either the UI or POST Product API. |
| SIG-BUS-0071              | PCFData Id does not exist                                                                                                                                                                                                |
| SIG-BUS-0072              | Given componentSupplierLink not exist                                                                                                                                                                                    |
| SIG-BUS-0073              | Error while parsing task service request details.                                                                                                                                                                        |
| SIG-BUS-0074              | Error while creating Pcf request.                                                                                                                                                                                        |
| SIG-BUS-0075              | Network SupplierConfiguration is not enabled.                                                                                                                                                                            |
| SIG-BUS-0076              | Invalid network provider, Please provide valid network provider.                                                                                                                                                         |
| SIG-BUS-0077              | Invalid network property `%s`, Please provide valid property for network provider `%s`.                                                                                                                                  |
| SIG-BUS-0078              | BPNL is already assigned to another supplier.                                                                                                                                                                            |
| SIG-BUS-0079              | Credential Id does not exist.                                                                                                                                                                                            |
| SIG-BUS-0080              | Invalid customer id.                                                                                                                                                                                                     |
| SIG-BUS-0081              | Product is not a root product.                                                                                                                                                                                           |
| SIG-BUS-0082              | Invalid Proof Id.                                                                                                                                                                                                        |
| SIG-BUS-0083              | Product not found.                                                                                                                                                                                                       |
| SIG-BUS-0084              | PCF Form Type `%s` does not exist.                                                                                                                                                                                       |
| SIG-BUS-0085              | PCF Form Standard Type `%s` does not exist.                                                                                                                                                                              |
| SIG-BUS-0086              | PCF Form LifecycleType Type `%s` does not exist.                                                                                                                                                                         |
| SIG-BUS-0087              | PCF Form DeclarationType Type {} does not exist.                                                                                                                                                                         |
| SIG-BUS-0088              | The cross sectoral standard selected is not supported by Catena.                                                                                                                                                         |
| SIG-BUS-0089              | Invalid taskId, there is no external request found for the given taskId.                                                                                                                                                 |
| SIG-BUS-0090              | No Share PCF Service found for taskType `%s`                                                                                                                                                                             |
| SIG-BUS-0091              | Catena-x is not enabled for your company                                                                                                                                                                                 |
| SIG-BUS-0092              | Maximum BOM limit reached. You Cannot add more BOM to this product.                                                                                                                                                      |
| SIG-BUS-0093              | Product Identifier is not a product                                                                                                                                                                                      |
| SIG-BUS-0094              | BOM is already created                                                                                                                                                                                                   |
| SIG-BUS-0095              | Component doesn't exist in the company.                                                                                                                                                                                  |
| SIG-BUS-0096              | Some technical issue occurred, please contact support team.                                                                                                                                                              |
| SIG-BUS-0097              | Task is already in `%s` status                                                                                                                                                                                           |
| SIG-BUS-0098              | Cannot perform operation with status `%s`                                                                                                                                                                                |
| SIG-BUS-0099              | Product not found for given product identifier.                                                                                                                                                                          |
| SIG-BUS-0100              | RequestId is already used, it should not be duplicate, please provide unique requestId.                                                                                                                                  |
| SIG-BUS-0102              | Product Credential with the given credentialId: `%s` and productId `%s` not found                                                                                                                                        |
| SIG-BUS-0121              | Summary file download failed.                                                                                                                                                                                            |
| SIG-BUS-0122              | Error occurred while scanning the file uploaded                                                                                                                                                                          |
| SIG-BUS-0124              | Company with ID {0} does not exist.                                                                                                                                                                                      |
| SIG-BUS-0125              | Company wallet is not found for company id:                                                                                                                                                                              |
| SIG-BUS-0127              | Job not found.                                                                                                                                                                                                           |
| SIG-BUS-0128              | Job Item is not available for jobItemUid: {0}.                                                                                                                                                                           |
| SIG-BUS-0129              | Product Emission Credential not available for jobItemUid: {0}, CompanyUid: {1}.                                                                                                                                          |
| SIG-BUS-0130              | Revoked Credential can not be activated.                                                                                                                                                                                 |
| SIG-BUS-0131              | Identifier types or values didn't match with component                                                                                                                                                                   |
| SIG-BUS-0132              | Supplier is not available in SiGREEN to send pcf request                                                                                                                                                                 |
| SIG-BUS-0133              | Cannot create task to share as we are getting multiple reference task                                                                                                                                                    |
| SIG-BUS-0134              | Cannot create task to share as we are not getting any reference task                                                                                                                                                     |
| SIG-BUS-0135              | Supplier status is not INVITED, cannot cancel invite                                                                                                                                                                     |
| SIG-BUS-0136              | Supplier already invited                                                                                                                                                                                                 |
| SIG-BUS-0137              | Supplier already active                                                                                                                                                                                                  |
| SIG-BUS-0138              | File type mismatched.                                                                                                                                                                                                    |
| SIG-BUS-0139              | Your account has reached the maximum allowed number of factories, which is `%s`. Additional factories cannot be added.                                                                                                   |
| SIG-BUS-0140              | Task with the given id (`%s`) cannot be found or it belongs to another company                                                                                                                                           |
| SIG-BUS-0141              | Only PCF request and PCF data share task type and InProgress or withdrawn task can be unlinked                                                                                                                           |
| SIG-BUS-0142              | Product cannot be found with the given id (`%s`) or it is not bound to the given task. (`%s`)                                                                                                                            |
| SIG-BUS-0143              | Product (`%s`) cannot be linked to the given task, because the product is deleted.                                                                                                                                       |
| SIG-BUS-0144              | Product (`%s`) cannot be linked to the given task, because it's not root product.                                                                                                                                        |
| SIG-BUS-0145              | Task (`%s`) is linked to a different product. Please unlink first!                                                                                                                                                       |
| SIG-BUS-0146              | Technical user not found for client id.                                                                                                                                                                                  |
| SIG-BUS-0147              | An unexpected error occurred.                                                                                                                                                                                            |
| SIG-BUS-0148              | Technical user not found.                                                                                                                                                                                                |
| SIG-BUS-0149              | The number of possible technical users per company was exceeded.                                                                                                                                                         |
| SIG-BUS-0150              | You are not authorized to perform the current operation.                                                                                                                                                                 |
| SIG-BUS-0151              | API Management for company not found.                                                                                                                                                                                    |
| SIG-BUS-0152              | API Management already enabled for company.                                                                                                                                                                              |
| SIG-BUS-0153              | API Package does not exist.                                                                                                                                                                                              |
| SIG-BUS-0154              | API Management needs to be enabled.                                                                                                                                                                                      |
| SIG-BUS-0155              | API Management already revoked for company.                                                                                                                                                                              |
| SIG-BUS-0156              | API Management needs to be revoked.                                                                                                                                                                                      |
| SIG-BUS-0157              | API Management cannot be revoked to perform this operation.                                                                                                                                                              |
| SIG-BUS-0158              | Permission Group is not available.                                                                                                                                                                                       |
| SIG-BUS-0159              | Technical User cannot be revoked to perform this operation.                                                                                                                                                              |
| SIG-BUS-0160              | Retries exceeded for fetching SiemensID token.                                                                                                                                                                           |
| SIG-BUS-0161              | BOM does not exists in company.                                                                                                                                                                                          |
| SIG-BUS-0162              | `%s` plan is already enabled for the company                                                                                                                                                                             |
| SIG-BUS-0163              | Invalid product identifier type.                                                                                                                                                                                         |
| SIG-BUS-0164              | Identifier value is required when identifier type is provided                                                                                                                                                            |
| SIG-BUS-0165              | Material category should be empty for UNCATEGORIZED component category                                                                                                                                                   |
| SIG-BUS-0166              | Factory emission does not exists.                                                                                                                                                                                        |
| SIG-BUS-0167              | Cannot unlink product, Product Id given for unlinking is not matching with already linked product                                                                                                                        |
| SIG-BUS-0168              | Cannot link product as task is already linked to another product                                                                                                                                                         |
| SIG-BUS-0169              | Product can be linked or unlinked from the task only if the status is in NEW, ASSIGNED OR WITHDRAWN status                                                                                                               |
| SIG-BUS-0170              | The material category (`%s`) already exists and is not accessible.                                                                                                                                                       |
| SIG-BUS-0171              | PACT is not enabled for your company                                                                                                                                                                                     |
| SIG-BUS-0172              | BOM version (`%s`) already exist.                                                                                                                                                                                        |
| SIG-BUS-0173              | BOM version (`%s`) does not exist.                                                                                                                                                                                       |
| SIG-BUS-0174              | Incorrect PACT network properties `%s` found.                                                                                                                                                                            |
| SIG-BUS-0175              | PACT Partner Network Configuration is already assigned to another supplier.                                                                                                                                              |
| SIG-BUS-0176              | BOM item does not exist.                                                                                                                                                                                                 |
| SIG-BUS-0177              | Cannot use parent product identifier as BOM items.                                                                                                                                                                       |
| SIG-BUS-0178              | Component does not exist `%s`.                                                                                                                                                                                           |
| SIG-BUS-0179              | Product does not exist `%s`.                                                                                                                                                                                             |
| SIG-BUS-0180              | New items can only be added under a group,component or emission item.                                                                                                                                                    |
| SIG-BUS-0181              | Locked BOM cannot be deleted.                                                                                                                                                                                            |
| SIG-BUS-0182              | Partner network configuration does not exist.                                                                                                                                                                            |
| SIG-BUS-0183              | BOM is already locked.                                                                                                                                                                                                   |
| SIG-BUS-0184              | The partner configuration is already mapped to a partner.                                                                                                                                                                |
| SIG-BUS-0185              | Connectivity properties are missing. Please update the connectivity properties.                                                                                                                                          |
| SIG-BUS-0186              | A job is still pending, cannot start new calculation.                                                                                                                                                                    |
| SIG-BUS-0187              | The cross sectoral standard selected is not supported by PACT.                                                                                                                                                           |
| SIG-BUS-0188              | The product family (`%s`) already exists and is not accessible.                                                                                                                                                          |
| SIG-BUS-0189              | PACT feature is already enabled for the company                                                                                                                                                                          |
| SIG-BUS-0190              | Maximum BOM item limit reached.                                                                                                                                                                                          |
| SIG-BUS-0191              | Maximum BOM nested level limit reached.                                                                                                                                                                                  |
| SIG-BUS-0192              | Maximum BOM version limit reached.                                                                                                                                                                                       |
| SIG-BUS-0193              | Request event id not found                                                                                                                                                                                               |
| SIG-BUS-0194              | PCF already shared for the given request event id                                                                                                                                                                        |
| SIG-BUS-0195              | Unit mismatch: Please ensure PCF values use compatible units.                                                                                                                                                            |
| SIG-BUS-0196              | Upstream emission data not available for the given BOM.                                                                                                                                                                  |
| SIG-BUS-0197              | Internal usage packages can not be enabled.                                                                                                                                                                              |
| SIG-BUS-0198              | BOM is empty, cannot proceed.                                                                                                                                                                                            |
| SIG-BUS-0199              | Cannot link an unrelated supplier.                                                                                                                                                                                       |
| SIG-BUS-0200              | The companies are already connected.                                                                                                                                                                                     |
| SIG-BUS-0201              | Requested connection request already exists.                                                                                                                                                                             |
| SIG-BUS-0202              | The connection request is already canceled.                                                                                                                                                                              |
| SIG-BUS-0203              | The connection request is already accepted.                                                                                                                                                                              |
| SIG-BUS-0204              | The connection request is already rejected.                                                                                                                                                                              |
| SIG-BUS-0205              | No connection request to cancel.                                                                                                                                                                                         |
| SIG-BUS-0206              | No connection request to reject.                                                                                                                                                                                         |
| SIG-BUS-0207              | No connection request to accept.                                                                                                                                                                                         |
| SIG-BUS-0208              | BOM item `%s` does not exist.                                                                                                                                                                                            |
| SIG-BUS-0209              | Company image does not exist.                                                                                                                                                                                            |
| SIG-BUS-0210              | Only connected companies can send message.                                                                                                                                                                               |
| SIG-BUS-0211              | Cannot delete credential because auto proof sharing is enabled for this credential.                                                                                                                                      |
| SIG-BUS-0213              | Supplier location cannot be changed.                                                                                                                                                                                     |
| SIG-BUS-0214              | Invalid or unsupported emission item(s) found in request.                                                                                                                                                                |

# SiGREEN My Tasks API – API Overview

## My Tasks API (SiGREEN Product version 1.16.0)

[Download OpenAPI Specification](api_specs/openapi-spec-v1.16.0.yaml)

## My Tasks API (Previous SiGREEN Product version 1.15.0)

[Download OpenAPI Specification](api_specs/openapi-spec-v1.15.0.yaml)

# Changelog

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).

## [1.15.0] - 2025-06-23

#### Breaking Changes

______________________________________________________________________

##### `GET` /tasks/inboundPcfRequests

###### Return Type:

Changed content type : `application/json`

```text
* Changed property `items` (array)

    Changed items (object):

    * Added property `productId` (string)
        > Linked Product, UUID to identify the product of the company in SiGREEN


    * Changed property `id` (string)
        > Task UUID to identify the task of the company in SiGREEN.


    * Changed property `taskId` (string)
        > Task id.


    * Changed property `productName` (string)
        > Linked Product, Name of the Product


    * Changed property `taskType` (string)
        > Task types from company.


    * Changed property `status` (string)
        > Task status.


        Added enum value:

        * `ASSIGNED`
    * Changed property `requestTimeStamp` (string)
        > PCF Request created DateTime. DateTime (ISO 8601; UTC Timezone)


    * Changed property `lastUpdatedTimeStamp` (string)
        > Last update DateTime. DateTime (ISO 8601; UTC Timezone)


    * Changed property `productFamily` (string)
        > Name of the Product Family linked to the Product


    * Changed property `pcfData` (object)

        * Changed property `pcfId` (string)
            > PCF ID of PCF data.


    * Changed property `pcfRequestedData` (object)

        * Added property `geographySubRegion` (string)
            > Expecting pcf data for the declared geography sub region.


            Enum values:

            * `Africa`
            * `Americas`
            * `Asia`
            * `Europe`
            * `Oceania`
            * `Australia and New Zealand`
            * `Central Asia`
            * `Eastern Asia`
            * `Eastern Europe`
            * `Latin America and the Caribbean`
            * `Melanesia`
            * `Micronesia`
            * `Northern Africa`
            * `Northern America`
            * `Northern Europe`
            * `Polynesia`
            * `South-eastern Asia`
            * `Southern Asia`
            * `Southern Europe`
            * `Sub-Saharan Africa`
            * `Western Asia`
            * `Western Europe`
            * `Global`
        * Added property `productName` (string)
            > Supplier Product Name regarding the product pcf data to be shared.

```

#### What's Changed

______________________________________________________________________

##### `PUT` /tasks/{id}/linkProduct/{productId}

###### Return Type:

Changed response : **400 Bad Request**

> Bad request.

- Changed content type : `application/problem+json`

  - Changed property `type` (string)

    > A URI reference that identifies the problem type. This should be a human-readable documentation for the problem type (e.g. list of SiGREEN Validation errors). When this member is not present, its value is assumed to be 'about:blank'.

  - Changed property `title` (string)

    > Developer friendly, human-readable description or summary that SHOULD NOT change from occurrence to occurrence of the problem. In case of using a custom HTTP status codes in the status field of the error object, the title MUST provide a concise description of the HTTP error code (for example: status: '599', title: 'Network Connection Timeout').

  - Changed property `errors` (array)

    > Extended error array to include descriptions about errors like validation errors.

    Changed items (object):

    - Changed property `code` (string)
      > String with a generated pointer to the wrong value; Collections integrated as an integer, starts with 0.

Changed response : **401 Unauthorized**

> Unauthorized.

- Changed content type : `application/problem+json`

  - Changed property `type` (string)

    > A URI reference that identifies the problem type. This should be a human-readable documentation for the problem type (e.g. list of SiGREEN Validation errors). When this member is not present, its value is assumed to be 'about:blank'.

  - Changed property `title` (string)

    > Developer friendly, human-readable description or summary that SHOULD NOT change from occurrence to occurrence of the problem. In case of using a custom HTTP status codes in the status field of the error object, the title MUST provide a concise description of the HTTP error code (for example: status: '599', title: 'Network Connection Timeout').

  - Changed property `errors` (array)

    > Extended error array to include descriptions about errors like validation errors.

    Changed items (object):

    - Changed property `code` (string)
      > String with a generated pointer to the wrong value; Collections integrated as an integer, starts with 0.

Changed response : **403 Forbidden**

> Forbidden.

- Changed content type : `application/problem+json`

  - Changed property `type` (string)

    > A URI reference that identifies the problem type. This should be a human-readable documentation for the problem type (e.g. list of SiGREEN Validation errors). When this member is not present, its value is assumed to be 'about:blank'.

  - Changed property `title` (string)

    > Developer friendly, human-readable description or summary that SHOULD NOT change from occurrence to occurrence of the problem. In case of using a custom HTTP status codes in the status field of the error object, the title MUST provide a concise description of the HTTP error code (for example: status: '599', title: 'Network Connection Timeout').

  - Changed property `errors` (array)

    > Extended error array to include descriptions about errors like validation errors.

    Changed items (object):

    - Changed property `code` (string)
      > String with a generated pointer to the wrong value; Collections integrated as an integer, starts with 0.

Changed response : \*\*5XX \*\*

> Unexpected error.

- Changed content type : `application/problem+json`

  - Changed property `type` (string)

    > A URI reference that identifies the problem type. This should be a human-readable documentation for the problem type (e.g. list of SiGREEN Validation errors). When this member is not present, its value is assumed to be 'about:blank'.

  - Changed property `title` (string)

    > Developer friendly, human-readable description or summary that SHOULD NOT change from occurrence to occurrence of the problem. In case of using a custom HTTP status codes in the status field of the error object, the title MUST provide a concise description of the HTTP error code (for example: status: '599', title: 'Network Connection Timeout').

  - Changed property `errors` (array)

    > Extended error array to include descriptions about errors like validation errors.

    Changed items (object):

    - Changed property `code` (string)
      > String with a generated pointer to the wrong value; Collections integrated as an integer, starts with 0.

##### `DELETE` /tasks/{id}/linkProduct/{productId}

###### Return Type:

Changed response : **400 Bad Request**

> Bad request.

- Changed content type : `application/problem+json`

  - Changed property `type` (string)

    > A URI reference that identifies the problem type. This should be a human-readable documentation for the problem type (e.g. list of SiGREEN Validation errors). When this member is not present, its value is assumed to be 'about:blank'.

  - Changed property `title` (string)

    > Developer friendly, human-readable description or summary that SHOULD NOT change from occurrence to occurrence of the problem. In case of using a custom HTTP status codes in the status field of the error object, the title MUST provide a concise description of the HTTP error code (for example: status: '599', title: 'Network Connection Timeout').

  - Changed property `errors` (array)

    > Extended error array to include descriptions about errors like validation errors.

    Changed items (object):

    - Changed property `code` (string)
      > String with a generated pointer to the wrong value; Collections integrated as an integer, starts with 0.

Changed response : **401 Unauthorized**

> Unauthorized.

- Changed content type : `application/problem+json`

  - Changed property `type` (string)

    > A URI reference that identifies the problem type. This should be a human-readable documentation for the problem type (e.g. list of SiGREEN Validation errors). When this member is not present, its value is assumed to be 'about:blank'.

  - Changed property `title` (string)

    > Developer friendly, human-readable description or summary that SHOULD NOT change from occurrence to occurrence of the problem. In case of using a custom HTTP status codes in the status field of the error object, the title MUST provide a concise description of the HTTP error code (for example: status: '599', title: 'Network Connection Timeout').

  - Changed property `errors` (array)

    > Extended error array to include descriptions about errors like validation errors.

    Changed items (object):

    - Changed property `code` (string)
      > String with a generated pointer to the wrong value; Collections integrated as an integer, starts with 0.

Changed response : **403 Forbidden**

> Forbidden.

- Changed content type : `application/problem+json`

  - Changed property `type` (string)

    > A URI reference that identifies the problem type. This should be a human-readable documentation for the problem type (e.g. list of SiGREEN Validation errors). When this member is not present, its value is assumed to be 'about:blank'.

  - Changed property `title` (string)

    > Developer friendly, human-readable description or summary that SHOULD NOT change from occurrence to occurrence of the problem. In case of using a custom HTTP status codes in the status field of the error object, the title MUST provide a concise description of the HTTP error code (for example: status: '599', title: 'Network Connection Timeout').

  - Changed property `errors` (array)

    > Extended error array to include descriptions about errors like validation errors.

    Changed items (object):

    - Changed property `code` (string)
      > String with a generated pointer to the wrong value; Collections integrated as an integer, starts with 0.

Changed response : \*\*5XX \*\*

> Unexpected error.

- Changed content type : `application/problem+json`

  - Changed property `type` (string)

    > A URI reference that identifies the problem type. This should be a human-readable documentation for the problem type (e.g. list of SiGREEN Validation errors). When this member is not present, its value is assumed to be 'about:blank'.

  - Changed property `title` (string)

    > Developer friendly, human-readable description or summary that SHOULD NOT change from occurrence to occurrence of the problem. In case of using a custom HTTP status codes in the status field of the error object, the title MUST provide a concise description of the HTTP error code (for example: status: '599', title: 'Network Connection Timeout').

  - Changed property `errors` (array)

    > Extended error array to include descriptions about errors like validation errors.

    Changed items (object):

    - Changed property `code` (string)
      > String with a generated pointer to the wrong value; Collections integrated as an integer, starts with 0.

##### `PUT` /tasks/{id}/sharePcf/{pcfId}

###### Parameters:

Changed: `pcfId` in `path`

> UUID of the PCF to identify the pcf data of the company in SiGREEN.

###### Return Type:

Changed response : **400 Bad Request**

> Bad request.

- Changed content type : `application/problem+json`

  - Changed property `type` (string)

    > A URI reference that identifies the problem type. This should be a human-readable documentation for the problem type (e.g. list of SiGREEN Validation errors). When this member is not present, its value is assumed to be 'about:blank'.

  - Changed property `title` (string)

    > Developer friendly, human-readable description or summary that SHOULD NOT change from occurrence to occurrence of the problem. In case of using a custom HTTP status codes in the status field of the error object, the title MUST provide a concise description of the HTTP error code (for example: status: '599', title: 'Network Connection Timeout').

  - Changed property `errors` (array)

    > Extended error array to include descriptions about errors like validation errors.

    Changed items (object):

    - Changed property `code` (string)
      > String with a generated pointer to the wrong value; Collections integrated as an integer, starts with 0.

Changed response : **401 Unauthorized**

> Unauthorized.

- Changed content type : `application/problem+json`

  - Changed property `type` (string)

    > A URI reference that identifies the problem type. This should be a human-readable documentation for the problem type (e.g. list of SiGREEN Validation errors). When this member is not present, its value is assumed to be 'about:blank'.

  - Changed property `title` (string)

    > Developer friendly, human-readable description or summary that SHOULD NOT change from occurrence to occurrence of the problem. In case of using a custom HTTP status codes in the status field of the error object, the title MUST provide a concise description of the HTTP error code (for example: status: '599', title: 'Network Connection Timeout').

  - Changed property `errors` (array)

    > Extended error array to include descriptions about errors like validation errors.

    Changed items (object):

    - Changed property `code` (string)
      > String with a generated pointer to the wrong value; Collections integrated as an integer, starts with 0.

Changed response : **403 Forbidden**

> Forbidden.

- Changed content type : `application/problem+json`

  - Changed property `type` (string)

    > A URI reference that identifies the problem type. This should be a human-readable documentation for the problem type (e.g. list of SiGREEN Validation errors). When this member is not present, its value is assumed to be 'about:blank'.

  - Changed property `title` (string)

    > Developer friendly, human-readable description or summary that SHOULD NOT change from occurrence to occurrence of the problem. In case of using a custom HTTP status codes in the status field of the error object, the title MUST provide a concise description of the HTTP error code (for example: status: '599', title: 'Network Connection Timeout').

  - Changed property `errors` (array)

    > Extended error array to include descriptions about errors like validation errors.

    Changed items (object):

    - Changed property `code` (string)
      > String with a generated pointer to the wrong value; Collections integrated as an integer, starts with 0.

Changed response : \*\*5XX \*\*

> Unexpected error.

- Changed content type : `application/problem+json`

  - Changed property `type` (string)

    > A URI reference that identifies the problem type. This should be a human-readable documentation for the problem type (e.g. list of SiGREEN Validation errors). When this member is not present, its value is assumed to be 'about:blank'.

  - Changed property `title` (string)

    > Developer friendly, human-readable description or summary that SHOULD NOT change from occurrence to occurrence of the problem. In case of using a custom HTTP status codes in the status field of the error object, the title MUST provide a concise description of the HTTP error code (for example: status: '599', title: 'Network Connection Timeout').

  - Changed property `errors` (array)

    > Extended error array to include descriptions about errors like validation errors.

    Changed items (object):

    - Changed property `code` (string)
      > String with a generated pointer to the wrong value; Collections integrated as an integer, starts with 0.

##### `PATCH` /tasks/{id}/pcf/accept

> Accept proof received to the task in SiGREEN.

###### Return Type:

Changed response : **400 Bad Request**

> Bad request.

- Changed content type : `application/problem+json`

  - Changed property `type` (string)

    > A URI reference that identifies the problem type. This should be a human-readable documentation for the problem type (e.g. list of SiGREEN Validation errors). When this member is not present, its value is assumed to be 'about:blank'.

  - Changed property `title` (string)

    > Developer friendly, human-readable description or summary that SHOULD NOT change from occurrence to occurrence of the problem. In case of using a custom HTTP status codes in the status field of the error object, the title MUST provide a concise description of the HTTP error code (for example: status: '599', title: 'Network Connection Timeout').

  - Changed property `errors` (array)

    > Extended error array to include descriptions about errors like validation errors.

    Changed items (object):

    - Changed property `code` (string)
      > String with a generated pointer to the wrong value; Collections integrated as an integer, starts with 0.

Changed response : **401 Unauthorized**

> Unauthorized.

- Changed content type : `application/problem+json`

  - Changed property `type` (string)

    > A URI reference that identifies the problem type. This should be a human-readable documentation for the problem type (e.g. list of SiGREEN Validation errors). When this member is not present, its value is assumed to be 'about:blank'.

  - Changed property `title` (string)

    > Developer friendly, human-readable description or summary that SHOULD NOT change from occurrence to occurrence of the problem. In case of using a custom HTTP status codes in the status field of the error object, the title MUST provide a concise description of the HTTP error code (for example: status: '599', title: 'Network Connection Timeout').

  - Changed property `errors` (array)

    > Extended error array to include descriptions about errors like validation errors.

    Changed items (object):

    - Changed property `code` (string)
      > String with a generated pointer to the wrong value; Collections integrated as an integer, starts with 0.

Changed response : **403 Forbidden**

> Forbidden.

- Changed content type : `application/problem+json`

  - Changed property `type` (string)

    > A URI reference that identifies the problem type. This should be a human-readable documentation for the problem type (e.g. list of SiGREEN Validation errors). When this member is not present, its value is assumed to be 'about:blank'.

  - Changed property `title` (string)

    > Developer friendly, human-readable description or summary that SHOULD NOT change from occurrence to occurrence of the problem. In case of using a custom HTTP status codes in the status field of the error object, the title MUST provide a concise description of the HTTP error code (for example: status: '599', title: 'Network Connection Timeout').

  - Changed property `errors` (array)

    > Extended error array to include descriptions about errors like validation errors.

    Changed items (object):

    - Changed property `code` (string)
      > String with a generated pointer to the wrong value; Collections integrated as an integer, starts with 0.

Changed response : \*\*5XX \*\*

> Unexpected error.

- Changed content type : `application/problem+json`

  - Changed property `type` (string)

    > A URI reference that identifies the problem type. This should be a human-readable documentation for the problem type (e.g. list of SiGREEN Validation errors). When this member is not present, its value is assumed to be 'about:blank'.

  - Changed property `title` (string)

    > Developer friendly, human-readable description or summary that SHOULD NOT change from occurrence to occurrence of the problem. In case of using a custom HTTP status codes in the status field of the error object, the title MUST provide a concise description of the HTTP error code (for example: status: '599', title: 'Network Connection Timeout').

  - Changed property `errors` (array)

    > Extended error array to include descriptions about errors like validation errors.

    Changed items (object):

    - Changed property `code` (string)
      > String with a generated pointer to the wrong value; Collections integrated as an integer, starts with 0.

##### `PATCH` /tasks/{id}/pcf/reject

> Reject proof received to the task in SiGREEN.

###### Request:

Changed content type : `application/json`

- Changed property `reason` (string)
  > Deny reason.

###### Return Type:

Changed response : **400 Bad Request**

> Bad request.

- Changed content type : `application/problem+json`

  - Changed property `type` (string)

    > A URI reference that identifies the problem type. This should be a human-readable documentation for the problem type (e.g. list of SiGREEN Validation errors). When this member is not present, its value is assumed to be 'about:blank'.

  - Changed property `title` (string)

    > Developer friendly, human-readable description or summary that SHOULD NOT change from occurrence to occurrence of the problem. In case of using a custom HTTP status codes in the status field of the error object, the title MUST provide a concise description of the HTTP error code (for example: status: '599', title: 'Network Connection Timeout').

  - Changed property `errors` (array)

    > Extended error array to include descriptions about errors like validation errors.

    Changed items (object):

    - Changed property `code` (string)
      > String with a generated pointer to the wrong value; Collections integrated as an integer, starts with 0.

Changed response : **401 Unauthorized**

> Unauthorized.

- Changed content type : `application/problem+json`

  - Changed property `type` (string)

    > A URI reference that identifies the problem type. This should be a human-readable documentation for the problem type (e.g. list of SiGREEN Validation errors). When this member is not present, its value is assumed to be 'about:blank'.

  - Changed property `title` (string)

    > Developer friendly, human-readable description or summary that SHOULD NOT change from occurrence to occurrence of the problem. In case of using a custom HTTP status codes in the status field of the error object, the title MUST provide a concise description of the HTTP error code (for example: status: '599', title: 'Network Connection Timeout').

  - Changed property `errors` (array)

    > Extended error array to include descriptions about errors like validation errors.

    Changed items (object):

    - Changed property `code` (string)
      > String with a generated pointer to the wrong value; Collections integrated as an integer, starts with 0.

Changed response : **403 Forbidden**

> Forbidden.

- Changed content type : `application/problem+json`

  - Changed property `type` (string)

    > A URI reference that identifies the problem type. This should be a human-readable documentation for the problem type (e.g. list of SiGREEN Validation errors). When this member is not present, its value is assumed to be 'about:blank'.

  - Changed property `title` (string)

    > Developer friendly, human-readable description or summary that SHOULD NOT change from occurrence to occurrence of the problem. In case of using a custom HTTP status codes in the status field of the error object, the title MUST provide a concise description of the HTTP error code (for example: status: '599', title: 'Network Connection Timeout').

  - Changed property `errors` (array)

    > Extended error array to include descriptions about errors like validation errors.

    Changed items (object):

    - Changed property `code` (string)
      > String with a generated pointer to the wrong value; Collections integrated as an integer, starts with 0.

Changed response : \*\*5XX \*\*

> Unexpected error.

- Changed content type : `application/problem+json`

  - Changed property `type` (string)

    > A URI reference that identifies the problem type. This should be a human-readable documentation for the problem type (e.g. list of SiGREEN Validation errors). When this member is not present, its value is assumed to be 'about:blank'.

  - Changed property `title` (string)

    > Developer friendly, human-readable description or summary that SHOULD NOT change from occurrence to occurrence of the problem. In case of using a custom HTTP status codes in the status field of the error object, the title MUST provide a concise description of the HTTP error code (for example: status: '599', title: 'Network Connection Timeout').

  - Changed property `errors` (array)

    > Extended error array to include descriptions about errors like validation errors.

    Changed items (object):

    - Changed property `code` (string)
      > String with a generated pointer to the wrong value; Collections integrated as an integer, starts with 0.

##### `GET` /tasks/inboundPcfRequests

> GET the incoming pcf requests of my company. It is recommended to make use of the "status" query parameter as follows /tasks/inboundPcfRequests?status=NEW,PCF_SHARED,COMPLETED.

###### Parameters:

Changed: `searchText` in `query`

> Free text search for incoming pcf requests that will look in the attribute of Task ID, Product name, Company Name (Requester company name) of a company in SiGREEN

Changed: `status` in `query`

> Inbound pcf request task status. Multiple fields could be used separated by commas (e.g. "NEW","PCF_SHARED").

Changed: `from` in `query`

> The start date of the date range during which the task was last modified in SiGREEN. DateTime (ISO 8601; UTC Timezone)

Changed: `to` in `query`

> The end date of the date range during which the task was last modified in SiGREEN. DateTime (ISO 8601; UTC Timezone)

Changed: `sort` in `query`

> The order of returned elements. Sort supporting fields (taskId, productName, companyName) for inbound pcf requests. Descending order could be requested by appending '-' ,ascending '+' at the prefix of parameter. (e.g. '+taskId')

Changed: `page` in `query`

> This is an integer representing the page-number to fetch. 0 would be the first page. (e.g 'page=0')

###### Return Type:

Changed response : **400 Bad Request**

> Bad request.

- Changed content type : `application/problem+json`

  - Changed property `type` (string)

    > A URI reference that identifies the problem type. This should be a human-readable documentation for the problem type (e.g. list of SiGREEN Validation errors). When this member is not present, its value is assumed to be 'about:blank'.

  - Changed property `title` (string)

    > Developer friendly, human-readable description or summary that SHOULD NOT change from occurrence to occurrence of the problem. In case of using a custom HTTP status codes in the status field of the error object, the title MUST provide a concise description of the HTTP error code (for example: status: '599', title: 'Network Connection Timeout').

  - Changed property `errors` (array)

    > Extended error array to include descriptions about errors like validation errors.

    Changed items (object):

    - Changed property `code` (string)
      > String with a generated pointer to the wrong value; Collections integrated as an integer, starts with 0.

Changed response : **401 Unauthorized**

> Unauthorized.

- Changed content type : `application/problem+json`

  - Changed property `type` (string)

    > A URI reference that identifies the problem type. This should be a human-readable documentation for the problem type (e.g. list of SiGREEN Validation errors). When this member is not present, its value is assumed to be 'about:blank'.

  - Changed property `title` (string)

    > Developer friendly, human-readable description or summary that SHOULD NOT change from occurrence to occurrence of the problem. In case of using a custom HTTP status codes in the status field of the error object, the title MUST provide a concise description of the HTTP error code (for example: status: '599', title: 'Network Connection Timeout').

  - Changed property `errors` (array)

    > Extended error array to include descriptions about errors like validation errors.

    Changed items (object):

    - Changed property `code` (string)
      > String with a generated pointer to the wrong value; Collections integrated as an integer, starts with 0.

Changed response : **403 Forbidden**

> Forbidden.

- Changed content type : `application/problem+json`

  - Changed property `type` (string)

    > A URI reference that identifies the problem type. This should be a human-readable documentation for the problem type (e.g. list of SiGREEN Validation errors). When this member is not present, its value is assumed to be 'about:blank'.

  - Changed property `title` (string)

    > Developer friendly, human-readable description or summary that SHOULD NOT change from occurrence to occurrence of the problem. In case of using a custom HTTP status codes in the status field of the error object, the title MUST provide a concise description of the HTTP error code (for example: status: '599', title: 'Network Connection Timeout').

  - Changed property `errors` (array)

    > Extended error array to include descriptions about errors like validation errors.

    Changed items (object):

    - Changed property `code` (string)
      > String with a generated pointer to the wrong value; Collections integrated as an integer, starts with 0.

Changed response : \*\*5XX \*\*

> Unexpected error.

- Changed content type : `application/problem+json`

  - Changed property `type` (string)

    > A URI reference that identifies the problem type. This should be a human-readable documentation for the problem type (e.g. list of SiGREEN Validation errors). When this member is not present, its value is assumed to be 'about:blank'.

  - Changed property `title` (string)

    > Developer friendly, human-readable description or summary that SHOULD NOT change from occurrence to occurrence of the problem. In case of using a custom HTTP status codes in the status field of the error object, the title MUST provide a concise description of the HTTP error code (for example: status: '599', title: 'Network Connection Timeout').

  - Changed property `errors` (array)

    > Extended error array to include descriptions about errors like validation errors.

    Changed items (object):

    - Changed property `code` (string)
      > String with a generated pointer to the wrong value; Collections integrated as an integer, starts with 0.

Changed response : **200 OK**

> Get incoming pcf requests for company in SiGREEN

- Changed content type : `application/json`

  - Changed property `items` (array)

    Changed items (object):

    - Added property `productId` (string)

      > Linked Product, UUID to identify the product of the company in SiGREEN

    - Changed property `id` (string)

      > Task UUID to identify the task of the company in SiGREEN.

    - Changed property `taskId` (string)

      > Task id.

    - Changed property `productName` (string)

      > Linked Product, Name of the Product

    - Changed property `taskType` (string)

      > Task types from company.

    - Changed property `status` (string)

      > Task status.

      Added enum value:

      - `ASSIGNED` * Changed property `requestTimeStamp` (string)
        > PCF Request created DateTime. DateTime (ISO 8601; UTC Timezone)

    - Changed property `lastUpdatedTimeStamp` (string)

      > Last update DateTime. DateTime (ISO 8601; UTC Timezone)

    - Changed property `productFamily` (string)

      > Name of the Product Family linked to the Product

    - Changed property `pcfData` (object)

      - Changed property `pcfId` (string)
        > PCF ID of PCF data.

    - Changed property `pcfRequestedData` (object)

      - Added property `geographySubRegion` (string)

        > Expecting pcf data for the declared geography sub region.

        Enum values:

        - `Africa`
        - `Americas`
        - `Asia`
        - `Europe`
        - `Oceania`
        - `Australia and New Zealand`
        - `Central Asia`
        - `Eastern Asia`
        - `Eastern Europe`
        - `Latin America and the Caribbean`
        - `Melanesia`
        - `Micronesia`
        - `Northern Africa`
        - `Northern America`
        - `Northern Europe`
        - `Polynesia`
        - `South-eastern Asia`
        - `Southern Asia`
        - `Southern Europe`
        - `Sub-Saharan Africa`
        - `Western Asia`
        - `Western Europe`
        - `Global` * Added property `productName` (string)
          > Supplier Product Name regarding the product pcf data to be shared.

##### `GET` /tasks/outboundPcfRequests

> GET the outbound pcf requests of my company. It is recommended to make use of the "status" query parameter as follows /tasks/outboundPcfRequests?status=NEW,PCF_RECEIVED,COMPLETED.

###### Parameters:

Changed: `searchText` in `query`

> Free text search for outbound pcf requests that will look in the attribute of Task ID, component name, supplierName of a company in SiGREEN.

Changed: `from` in `query`

> The start date of the date range during which the task was last modified in SiGREEN. DateTime (ISO 8601; UTC Timezone)

Changed: `to` in `query`

> The end date of the date range during which the task was last modified in SiGREEN. DateTime (ISO 8601; UTC Timezone)

Changed: `sort` in `query`

> The order of returned elements. Sort supporting fields (taskId, componentName, supplierName, supplierId, requestTimeStamp) in outbound pcf requests. Descending order could be requested by appending '-' ,ascending '+' at the prefix of parameter. (e.g. '+taskId')

Changed: `page` in `query`

> This is an integer representing the page-number to fetch. 0 would be the first page. (e.g 'page=0')

Changed: `size` in `query`

> This is used for the requested maximum page size. (e.g 'size=10')

###### Return Type:

Changed response : **400 Bad Request**

> Bad request.

- Changed content type : `application/problem+json`

  - Changed property `type` (string)

    > A URI reference that identifies the problem type. This should be a human-readable documentation for the problem type (e.g. list of SiGREEN Validation errors). When this member is not present, its value is assumed to be 'about:blank'.

  - Changed property `title` (string)

    > Developer friendly, human-readable description or summary that SHOULD NOT change from occurrence to occurrence of the problem. In case of using a custom HTTP status codes in the status field of the error object, the title MUST provide a concise description of the HTTP error code (for example: status: '599', title: 'Network Connection Timeout').

  - Changed property `errors` (array)

    > Extended error array to include descriptions about errors like validation errors.

    Changed items (object):

    - Changed property `code` (string)
      > String with a generated pointer to the wrong value; Collections integrated as an integer, starts with 0.

Changed response : **401 Unauthorized**

> Unauthorized.

- Changed content type : `application/problem+json`

  - Changed property `type` (string)

    > A URI reference that identifies the problem type. This should be a human-readable documentation for the problem type (e.g. list of SiGREEN Validation errors). When this member is not present, its value is assumed to be 'about:blank'.

  - Changed property `title` (string)

    > Developer friendly, human-readable description or summary that SHOULD NOT change from occurrence to occurrence of the problem. In case of using a custom HTTP status codes in the status field of the error object, the title MUST provide a concise description of the HTTP error code (for example: status: '599', title: 'Network Connection Timeout').

  - Changed property `errors` (array)

    > Extended error array to include descriptions about errors like validation errors.

    Changed items (object):

    - Changed property `code` (string)
      > String with a generated pointer to the wrong value; Collections integrated as an integer, starts with 0.

Changed response : **403 Forbidden**

> Forbidden.

- Changed content type : `application/problem+json`

  - Changed property `type` (string)

    > A URI reference that identifies the problem type. This should be a human-readable documentation for the problem type (e.g. list of SiGREEN Validation errors). When this member is not present, its value is assumed to be 'about:blank'.

  - Changed property `title` (string)

    > Developer friendly, human-readable description or summary that SHOULD NOT change from occurrence to occurrence of the problem. In case of using a custom HTTP status codes in the status field of the error object, the title MUST provide a concise description of the HTTP error code (for example: status: '599', title: 'Network Connection Timeout').

  - Changed property `errors` (array)

    > Extended error array to include descriptions about errors like validation errors.

    Changed items (object):

    - Changed property `code` (string)
      > String with a generated pointer to the wrong value; Collections integrated as an integer, starts with 0.

Changed response : \*\*5XX \*\*

> Unexpected error.

- Changed content type : `application/problem+json`

  - Changed property `type` (string)

    > A URI reference that identifies the problem type. This should be a human-readable documentation for the problem type (e.g. list of SiGREEN Validation errors). When this member is not present, its value is assumed to be 'about:blank'.

  - Changed property `title` (string)

    > Developer friendly, human-readable description or summary that SHOULD NOT change from occurrence to occurrence of the problem. In case of using a custom HTTP status codes in the status field of the error object, the title MUST provide a concise description of the HTTP error code (for example: status: '599', title: 'Network Connection Timeout').

  - Changed property `errors` (array)

    > Extended error array to include descriptions about errors like validation errors.

    Changed items (object):

    - Changed property `code` (string)
      > String with a generated pointer to the wrong value; Collections integrated as an integer, starts with 0.

Changed response : **200 OK**

> Get outbound pcf requests for company in SiGREEN

- Changed content type : `application/json`

  - Changed property `items` (array)

    Changed items (object):

    - Changed property `id` (string)

      > Task UUID to identify the task of the company in SiGREEN.

    - Changed property `taskId` (string)

      > Task id.

    - Changed property `componentName` (string)

      > Component name.

    - Changed property `supplierName` (string)

      > Supplier company name.

    - Changed property `status` (string)

      > Task status.

    - Changed property `requestTimeStamp` (string)

      > PCF Request created DateTime. DateTime (ISO 8601; UTC Timezone)

    - Changed property `lastUpdatedTimeStamp` (string)

      > Last update DateTime. DateTime (ISO 8601; UTC Timezone)

    - Changed property `materialCategory` (string)

      > Material category of the component

    - Changed property `supplierId` (string)

      > Supplier ID.

    - Changed property `pcfData` (object)

      - Changed property `pcfId` (string)
        > PCF ID of PCF data.
      - Changed property `pcfState` (string)
        > State of the PCF to say if it is self-certified or certified by a 3rd party certifier
      - Changed property `standards` (array)
        > PCF format or standards of the PCF data shared.

    - Changed property `pcfRequestedData` (object)

      - Added property `geographySubRegion` (string)
        > Expecting pcf data for the declared geography sub region.
      - Added property `productName` (string)
        > Supplier Product Name regarding the product pcf data to be shared.

## [1.12.0] - 2025-02-10

#### What's New

______________________________________________________________________

##### `PUT` /tasks/{id}/linkProduct/{productId}

> Link product to the task.

##### `DELETE` /tasks/{id}/linkProduct/{productId}

> Delete link between product and task.

##### `PUT` /tasks/{id}/sharePcf/{pcfId}

> Share the pcf data with the given task.

##### `PATCH` /tasks/{id}/pcf/accept

> accept proof received to the task in SiGREEN.

##### `PATCH` /tasks/{id}/pcf/reject

> reject proof received to the task in SiGREEN.

#### What's Changed

______________________________________________________________________

##### `GET` /tasks/inboundPcfRequests

###### Parameters:

Added: `id` in `query`

> UUID of the Task for identifying the incoming pcf request of the company in SiGREEN

###### Return Type:

Deleted response : **204 No Content**

> No Content.The request is successfully fulfilled, but no additional content is provided in the response.

Changed response : **200 OK**

> Get incoming pcf requests for company in SiGREEN

- Changed content type : `application/json`

  - Changed property `items` (array)

  Changed items (object):

  ```text
  * Changed property `id` (string)
    > task UUID to identify the task of the company in SiGREEN.

  * Changed property `pcfRequestedData` (object)

      * Added property `supplierProductId` (string)
          > Requesting supplier ProductId as product identifier gives more information regarding the product pcf data to be shared.

      * Changed property `message` (string)
          > Requesting pcf data with message, which gives more information regarding the product pcf data to be shared.

      * Changed property `identifiers` (array)
          > Additional product identifiers gives more information regarding the product pcf data to be shared.

  ```

##### `GET` /tasks/outboundPcfRequests

###### Parameters:

Added: `id` in `query`

> UUID of the Task for identifying the outbound pcf request of the company in SiGREEN

###### Return Type:

Changed response : **200 OK**

> Get outbound pcf requests for company in SiGREEN

- Changed content type : `application/json`

  - Changed property `items` (array)

  Changed items (object):

  ```text
  * Changed property `id` (string)
    > task UUID to identify the task of the company in SiGREEN.

  * Changed property `taskId` (string)
      > task id.

  * Changed property `pcfData` (object)

      * Added property `pcfState` (string)
          > State of the pcf to say if it self certified or certified by a 3rd party certifier

  * Changed property `pcfRequestedData` (object)

      * Added property `supplierProductId` (string)
          > Requesting supplier ProductId as product identifier gives more information regarding the product pcf data to be shared.

      * Changed property `message` (string)
          > Requesting pcf data with message, which gives more information regarding the product pcf data to be shared.

      * Changed property `identifiers` (array)
          > Additional product identifiers gives more information regarding the product pcf data to be shared.

  ```

## [1.11.0] - 2024-07-22

### Initial Release

# Quick Start

For getting started with our Application and our APIs, kindly refer to [Getting Started](../getting-started.html).

For general information about the **My Tasks** functionality, refer to the respective section in the SiGREEN Help: [My Tasks](https://siemens-help.stonly.com/kb/guide/en/my-tasks-94x8Zx0r1b/Steps/3528617).

## Concepts and Glossary

For detailed information about the concepts and terminology, please refer to [Concepts and Glossary](../concepts-glossary.html).

## My Tasks API

With the available Components part of My Tasks APIs, you can perform the following operations:

Product Tasks:

1. GET the incoming pcf requests received for a company in SiGREEN.
1. Share the pcf data for the given task with a linked product.
1. Link an existing Product to the given task.
1. Delete the link between a product and the task. It is possible only before the task status is changed to **PCF Shared**.

Component Tasks:

1. GET the outbound requests sent for a company in SiGREEN.
1. Reject proof received by the task in SiGREEN.
1. Accept proof received by the task in SiGREEN.

Task Comments:

1. Get comments for the given task in SiGREEN.
1. Add a comment to the given task in SiGREEN.
1. Update a comment to the given task in SiGREEN.
1. Delete a comment to the given task in SiGREEN.

For more information in detail about the **Parameters**, **Request Body**, and **Responses** refer to the [API Reference](api-my-tasks.html) under My Tasks API(s).

# SiGREEN PCF API – API Overview

## PCF API (SiGREEN Product version 1.16.1)

[Download OpenAPI Specification](api_specs/openapi-spec-v1.16.1.yaml)

## PCF API (Previous SiGREEN Product version 1.16.0)

[Download OpenAPI Specification](api_specs/openapi-spec-v1.16.0.yaml)

# Changelog

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).

## [1.15.0] - 2025-06-23

#### Breaking Changes

______________________________________________________________________

##### `POST` /pcf/products/{id}/standard/pact

###### Request:

Changed content type : `application/json`

Updated `PACT-V2.2.0` : * Changed property `pcfData` (object)

```text
New required properties:
- `productCarbonContent`

```

#### What's Changed

______________________________________________________________________

##### `POST` /pcf/products/{id}/standard/tfs

###### Request:

Changed content type : `application/json`

Updated `TFS-V2` : * Changed property `pcfData` (object)

```text
* Changed property `allocationRulesDescription` (string)
    > Description of the allocation rules applied to multi-output processes in your foreground data and explain the underlying reasons (way of allocating all activities from your manufacturing steps to the declared unit). Description of the allocation rules applied to end-of-life processes (also refers to manufacturing related waste).


* Changed property `secondaryEmissionFactorSources` (array)
    > Secondary data sources and versions used for this declaration, e.g. data bases such as ecoinvent.


    Changed items (string):

```

##### `POST` /pcf/products/{id}/standard/pact

###### Request:

Changed content type : `application/json`

Updated `PACT-V2.2.0` : * Changed property `pcfData` (object)

```text
New required properties:
- `productCarbonContent`

```

## [1.13.0] - 2025-03-17

#### What's New

______________________________________________________________________

##### `POST` /pcf/products/{id}/standard/iso

> To create a ISO standard credential for a specific product of a company in SiGREEN.

##### `POST` /pcf/products/{id}/standard/pact

> To create a PACT standard credential for a specific product of a company in SiGREEN.

#### What's Changed

______________________________________________________________________

##### `POST` /pcf/products/{id}/standard/tfs

###### Request:

Changed content type : `application/json`

Updated `TFS-V2` : * Changed property `pcfData` (object)

```text
* Changed property `referencePeriodStart` (string -> string)
    > Start of time period of data collection for primary data sources (this does not refer to publication dates of secondary data). DateTime (ISO 8601; UTC Timezone)


* Changed property `referencePeriodEnd` (string -> string)
    > End of time period of data collection for primary data sources. DateTime (ISO 8601; UTC Timezone)


* Changed property `created` (string -> string)
    > The time stamp at which the PCF has been declared, independently of when or if it has been shared. This represents the validity period start unless specified separately. DateTime (ISO 8601; UTC Timezone)


* Changed property `validityPeriodStart` (string -> string)
    > The time stamp at which the PCF is declared valid. If not specified, the validity period starts with the date of issue. DateTime (ISO 8601; UTC Timezone)


* Changed property `validityPeriodEnd` (string -> string)
    > Time stamp declaring the expected end of the use period for this declaration or date of expected update (i.e. when does the data validity period end?). DateTime (ISO 8601; UTC Timezone)

```

##### `POST` /pcf/products/{id}/standard/catena-x

###### Request:

Changed content type : `application/json`

Updated `CATENA-X-V2.2` : * Changed property `pcfData` (object)

```text
* Changed property `referencePeriodStart` (string -> string)
    > Start of time period of data collection for primary data sources (this does not refer to publication dates of secondary data). DateTime (ISO 8601; UTC Timezone)


* Changed property `referencePeriodEnd` (string -> string)
    > End of time period of data collection for primary data sources. DateTime (ISO 8601; UTC Timezone)


* Changed property `created` (string -> string)
    > The time stamp at which the PCF has been declared, independently of when or if it has been shared. This represents the validity period start unless specified separately. DateTime (ISO 8601; UTC Timezone)


* Changed property `validityPeriodStart` (string -> string)
    > The time stamp at which the PCF is declared valid. If not specified, the validity period starts with the date of issue. DateTime (ISO 8601; UTC Timezone)


* Changed property `validityPeriodEnd` (string -> string)
    > Time stamp declaring the expected end of the use period for this declaration or date of expected update (i.e. when does the data validity period end?). DateTime (ISO 8601; UTC Timezone)

```

______________________________________________________________________

## [1.12.0] - 2024-02-10

#### What's New

______________________________________________________________________

##### `POST` /pcf/products/{id}/standard/tfs

> To create a TFS standard credential for a specific product of a company in SiGREEN.

##### `POST` /pcf/products/{id}/standard/catena-x

> To create a CatenaX standard credential for a specific product of a company in SiGREEN.

##### `GET` /pcf/jobs/{id}/status

> It is to know whether credential has been created or not.

#### What's Deleted

______________________________________________________________________

##### `POST` /products/{productId}

> To create a credential for a specific product in a company in SiGREEN.

##### `GET` /jobs/{jobType}/{jobId}/status

> It is to know whether a credential has been created or not.

## [1.10.0-beta] - 2024-04-29

### Initial Beta Release

# Quick Start

For getting started with our Application and our APIs, kindly refer to [Getting Started](../getting-started.html).

## Concepts and Glossary

For detailed information about the concepts and terminology, please refer to [Concepts and Glossary](../concepts-glossary.html).

## PCF API

The PCF APIs are used for creating the Product Carbon Footprint (PCF) of any product via the SiGREEN application. With the available PCF APIs, you can perform the following operations:

1. Create the TFS standard credential for a specific product of a company in SiGREEN.
1. Create the PACT standard credential for a specific product of a company in SiGREEN.
1. Create an ISO standard credential for a specific product of a company in SiGREEN.
1. Create the CatenaX standard credential for a specific product of a company in SiGREEN.
1. Fetch the status of a job. It is to know whether a credential has been created or not.

For more information in detail about the **Parameters**, **Request Body**, and **Responses** refer to the [API Reference](api-pcf.html) under PCF API(s).

# SiGREEN Procurement API – API Overview

## Procurement API (SiGREEN Product version 1.16.9)

[Download OpenAPI Specification](api_specs/openapi-spec-v1.16.9.yaml)

## Procurement API (SiGREEN Product version 1.16.0)

[Download OpenAPI Specification](api_specs/openapi-spec-v1.16.0.yaml)

## Procurement API (Previous SiGREEN Product version 1.15.0)

[Download OpenAPI Specification](api_specs/openapi-spec-v1.15.0.yaml)

# Changelog

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).

## [1.15.0] - 2025-06-23

#### Breaking Changes

______________________________________________________________________

##### `POST` /suppliers/identifierTypes

###### Request:

Changed content type : `application/json`

New required properties:

- `mandatory`

- Added property `mandatory` (boolean)

  > Whether this identifier is mandatory or not. Mandatory properties must be provided

note

Please note that this change will only affect new data. Existing data will not be affected unless they are updated.

##### `PATCH` /suppliers/identifierTypes/{id}

###### Request:

Changed content type : `application/json`

New required properties:

- `mandatory`

- Added property `mandatory` (boolean)

  > Whether this identifier is mandatory or not. Mandatory properties must be provided

note

Please note that this change will only affect new data. Existing data will not be affected unless they are updated.

##### `POST` /components/identifierTypes

###### Request:

Changed content type : `application/json`

New required properties:

- `mandatory`

- Added property `mandatory` (boolean)

  > Whether this identifier is mandatory or not. Mandatory properties must be provided

note

Please note that this change will only affect new data. Existing data will not be affected unless they are updated.

##### `PATCH` /components/identifierTypes/{id}

###### Request:

Changed content type : `application/json`

New required properties:

- `mandatory`

- Added property `mandatory` (boolean)

  > Whether this identifier is mandatory or not. Mandatory properties must be provided

note

Please note that this change will only affect new data. Existing data will not be affected unless they are updated.

#### What's Changed

______________________________________________________________________

##### `GET` /suppliers/identifierTypes

###### Return Type:

Changed response : **200 OK**

> List of supplier identifier types

- Changed content type : `application/json`

  - Changed property `items` (array)

    Changed items (object):

    New required properties:

    - `isMandatory`

    - Added property `isMandatory` (boolean)

      > Whether this identifier is mandatory or not. Mandatory properties must be provided

##### `POST` /suppliers/identifierTypes

###### Request:

Changed content type : `application/json`

New required properties:

- `mandatory`

- Added property `mandatory` (boolean)

  > Whether this identifier is mandatory or not. Mandatory properties must be provided

##### `PATCH` /suppliers/identifierTypes/{id}

###### Request:

Changed content type : `application/json`

New required properties:

- `mandatory`

- Added property `mandatory` (boolean)

  > Whether this identifier is mandatory or not. Mandatory properties must be provided

##### `GET` /suppliers

###### Parameters:

Deleted: `sigreenId` in `query`

> Supplier company SiGREEN Id. Available only in when supplier status is AVAILABLE.

###### Return Type:

Changed response : **200 OK**

> A list of supplier

- Changed content type : `application/json`

  - Changed property `items` (array)

    Changed items (object):

    - Changed property `sigreenId` (string)
      > SiGREEN Identifier is to be able to uniquely identify a supplier company(legal entities) globally within the SiGREEN platform. The SiGREEN ID will always be present for ACTIVE suppliers.

##### `POST` /suppliers

###### Request:

Changed content type : `application/json`

- Added property `sigreenId` (string)
  > SiGREEN Identifier is to be able to uniquely identify a supplier company(legal entities) globally within the SiGREEN platform. The SiGREEN ID can be updated only if the supplier status is not ACTIVE

##### `GET` /suppliers/{id}

###### Return Type:

Changed response : **200 OK**

> Supplier detail by id

- Changed content type : `application/json`

  - Changed property `sigreenId` (string)
    > SiGREEN Identifier is to be able to uniquely identify a supplier company(legal entities) globally within the SiGREEN platform. The SiGREEN ID will always be present for ACTIVE suppliers.

##### `PUT` /suppliers/{id}

###### Request:

Changed content type : `application/json`

- Added property `sigreenId` (string)
  > SiGREEN Identifier is to be able to uniquely identify a supplier company(legal entities) globally within the SiGREEN platform. The SiGREEN ID can be updated only if the supplier status is not ACTIVE

##### `GET` /components/identifierTypes

###### Return Type:

Changed response : **200 OK**

> List of component identifier types.

- Changed content type : `application/json`

  - Changed property `items` (array)

    Changed items (object):

    New required properties:

    - `isMandatory`

    - Added property `isMandatory` (boolean)

      > Whether this identifier is mandatory or not. Mandatory properties must be provided

##### `POST` /components/identifierTypes

###### Request:

Changed content type : `application/json`

New required properties:

- `mandatory`

- Added property `mandatory` (boolean)

  > Whether this identifier is mandatory or not. Mandatory properties must be provided

##### `PATCH` /components/identifierTypes/{id}

###### Request:

Changed content type : `application/json`

New required properties:

- `mandatory`

- Added property `mandatory` (boolean)

  > Whether this identifier is mandatory or not. Mandatory properties must be provided

##### `POST` /components/{id}/suppliers/{supplierId}/pcfRequest

###### Request:

Changed content type : `application/json`

- Added property `geographySubRegion` (string)

  > Preferred region sub region of PCF request.

  Enum values:

  - `Africa`
  - `Americas`
  - `Asia`
  - `Europe`
  - `Oceania`
  - `Australia and New Zealand`
  - `Central Asia`
  - `Eastern Asia`
  - `Eastern Europe`
  - `Latin America and the Caribbean`
  - `Melanesia`
  - `Micronesia`
  - `Northern Africa`
  - `Northern America`
  - `Northern Europe`
  - `Polynesia`
  - `South-eastern Asia`
  - `Southern Asia`
  - `Southern Europe`
  - `Sub-Saharan Africa`
  - `Western Asia`
  - `Western Europe`
  - `Global`
  - `Several`

## [1.14.0] - 2025-03-14

#### What's Changed

______________________________________________________________________

##### `GET` /suppliers

###### Parameters:

Added: `sigreenId` in `query`

> Supplier company SiGREEN Id. Available only in when supplier status is AVAILABLE.

###### Return Type:

Changed response : **200 OK**

> A list of supplier

- Changed content type : `application/json`

  - Changed property `items` (array)

    Changed items (object):

    - Added property `sigreenId` (string)
      > Supplier company SiGREEN Id. Available only in when supplier status is AVAILABLE.

##### `GET` /suppliers/{id}

###### Return Type:

Changed response : **200 OK**

> Supplier detail by id

- Changed content type : `application/json`

  - Added property `sigreenId` (string)
    > Supplier company SiGREEN Id. Available only in when supplier status is AVAILABLE.

## [1.13.0] - 2025-02-10

### What's New

______________________________________________________________________

##### `POST` /suppliers/{id}/invite

> Invite supplier to SiGREEN via email

##### `GET` /materialCategories

> Get all material categories for company in SiGREEN

##### `POST` /materialCategories

> Create material category in SiGREEN.

##### `GET` /materialCategories/{id}

> Get material category details by id

##### `PUT` /materialCategories/{id}

> Update material category details by id

##### `DELETE` /materialCategories/{id}

> Delete material categories details by id

##### `GET` /materialCategories/{id}/image

> Get material category image by id

##### `PUT` /materialCategories/{id}/image

> Update material category image by id

##### `DELETE` /materialCategories/{id}/image

> Delete material category image by id

##### `POST` /suppliers/{id}/invite/cancel

> Cancel supplier invite to SiGREEN

##### `GET` /components/averageEmissions

> Get an average emission for a component.

##### `DELETE` /suppliers/{id}

> Delete supplier by supplier Id in SiGREEN

##### `DELETE` /components/{id}

> Delete component by id

#### What's Deleted

______________________________________________________________________

##### `GET` /components/pcf

> Get an aggregated export of a PCF of a component without cryptographic material. If you need cryptographic verifiability of this information please use DIDComm APIs.

#### What's Changed

______________________________________________________________________

##### `POST` /suppliers/identifierTypes

###### Return Type:

Changed response : **201 Created**

> Created. Response will return the UUID of the created supplier identifier type.

- Changed content type : `application/json`

  - Changed property `id` (string)
    > UUID to identify the identifier type of the company in SiGREEN

##### `DELETE` /suppliers/identifierTypes/{id}

###### Parameters:

Changed: `id` in `path`

> UUID to identify the supplier identifier type of the company in SiGREEN.

##### `PATCH` /suppliers/identifierTypes/{id}

###### Parameters:

Changed: `id` in `path`

> UUID for identifying the supplier identifier type of the company in SiGREEN.

##### `GET` /suppliers

###### Parameters:

Added: `searchText` in `query`

> Partial text search on following attributes - locations , name , identifiers.\
> Minimum 3 characters required to search

Added: `idType` in `query`

> Supplier identifier type in SiGREEN.\
> idType filter is applicable only when user is searching with idValue filter.\
> Idea here is to find suppliers with idType and value combination.

Added: `idValue` in `query`

> Filter suppliers based on either main or additional identifier value in SiGREEN.

Added: `status` in `query`

> Supplier status

Added: `sort` in `query`

> Sorting fields supported (name, createdOn, updateOn). Example - Ascending order '+name' , Descending order '-name'. Default order is createdOn ascending order.

Deleted: `identifier` in `query`

> Main Supplier Identifier to identify the supplier of the company in SiGREEN

###### Return Type:

Changed response : **200 OK**

> A list of supplier

- Changed content type : `application/json`

  - Changed property `items` (array)

  Changed items (object):

  ```text
  * Added property `status` (string)
    > Supplier Status 
       
       NOT_INVITED_YET  - When supplier created 
       INVITED - When supplier invited 
       INVITE_CANCELLED - When invite cancelled 
       DENIED_BY_ADMIN - When system/company admin rejected user 
       AVAILABLE - When supplier in SiGREEN

      Enum values:

      * `NOT_INVITED_YET`
      * `INVITED`
      * `INVITE_CANCELLED`
      * `DENIED_BY_ADMIN`
      * `AVAILABLE`
  * Added property `createdOn` (string)
      > Created on (ISO 8601; UTC Timezone)

  * Added property `updatedOn` (string)
      > Updated on (ISO 8601; UTC Timezone)

  * Changed property `id` (string)
      > UUID to identify the supplier of the company in SiGREEN

  ```

##### `POST` /suppliers

###### Return Type:

Changed response : **201 Created**

> Created

- Changed content type : `application/json`

  - Changed property `id` (string)
    > UUID to identify the supplier of the company in SiGREEN

##### `GET` /suppliers/{id}

###### Parameters:

Changed: `id` in `path`

> UUID to identify the supplier of the company in SiGREEN

###### Return Type:

Changed response : **200 OK**

> Supplier detail by id

- Changed content type : `application/json`

  - Added property `status` (string)

    > Supplier Status
    >
    > NOT_INVITED_YET - When supplier created\
    > INVITED - When supplier invited\
    > INVITE_CANCELLED - When invite cancelled\
    > DENIED_BY_ADMIN - When system/company admin rejected user\
    > AVAILABLE - When supplier in SiGREEN

  - Added property `createdOn` (string)

    > Created on (ISO 8601; UTC Timezone)

  - Added property `updatedOn` (string)

    > Updated on (ISO 8601; UTC Timezone)

  - Changed property `id` (string)

    > UUID to identify the supplier of the company in SiGREEN

##### `PUT` /suppliers/{id}

###### Parameters:

Changed: `id` in `path`

> UUID to identify the supplier of the company in SiGREEN

##### `POST` /components/identifierTypes

###### Return Type:

Changed response : **201 Created**

> Created. Response will return the UUID of the created component identifier type.

- Changed content type : `application/json`

  - Changed property `id` (string)
    > UUID to identify the identifier type of the company in SiGREEN

##### `DELETE` /components/identifierTypes/{id}

###### Parameters:

Changed: `id` in `path`

> UUID to identify the component identifier type of the company in SiGREEN.

##### `PATCH` /components/identifierTypes/{id}

###### Parameters:

Changed: `id` in `path`

> UUID for identifying the component identifier type of the company in SiGREEN.

##### `GET` /components

###### Parameters:

Added: `searchText` in `query`

> Partial text search on following attributes - materialCategory , name , identifiers.\
> Search text minimum 3 characters required

Added: `componentsCategory` in `query`

> CATEGORIZED -> Components belongs to material category\
> uncategorized -> Components doesn't belongs to any material category\
> ALL -> All components

Added: `idType` in `query`

> Supplier identifier type in SiGREEN.\
> idType filter is applicable only when user is searching with idValue filter.\
> Idea here is to find components with idType and value combination.

Added: `idValue` in `query`

> Filter components based on either main or additional identifier value in SiGREEN.

Added: `sort` in `query`

> Sorting fields supported (name, createdOn, updateOn Example - Ascending order '+name' , Descending order '-name', Default order by createdOn ascending order

Deleted: `identifier` in `query`

> Main Component Identifier to identify the component of the company in SiGREEN

Changed: `materialCategory` in `query`

> Filter components by material category name.\
> Make sure don't use this filter with combination of componentsCategory as uncategorized

###### Return Type:

Changed response : **200 OK**

> A list of components

- Changed content type : `application/json`

  - Changed property `items` (array)

  Changed items (object):

  ```text
  * Added property `createdOn` (string)
    > Created on (ISO 8601; UTC Timezone)

  * Added property `updatedOn` (string)
      > Updated on (ISO 8601; UTC Timezone)

  * Changed property `id` (string)
      > UUID to identify the component of the company in SiGREEN

  * Changed property `unit` (string)
      > Component Unit type

  ```

##### `POST` /components

###### Request:

Changed content type : `application/json`

- Changed property `unit` (string)
  > Component Unit type

###### Return Type:

Changed response : **201 Created**

> Created

- Changed content type : `application/json`

  - Changed property `id` (string)
    > UUID to identify the component of the company in SiGREEN

##### `GET` /components/{id}

###### Parameters:

Changed: `id` in `path`

> UUID to identify the component of the company in SiGREEN

###### Return Type:

Changed response : **200 OK**

> A list of secondary data per component

- Changed content type : `application/json`

  - Added property `createdOn` (string)

    > Created on (ISO 8601; UTC Timezone)

  - Added property `updatedOn` (string)

    > Updated on (ISO 8601; UTC Timezone)

  - Changed property `id` (string)

    > UUID to identify the component of the company in SiGREEN

  - Changed property `unit` (string)

    > Component Unit type

##### `PUT` /components/{id}

###### Parameters:

Changed: `id` in `path`

> UUID to identify the component of the company in SiGREEN

###### Request:

Changed content type : `application/json`

- Changed property `unit` (string)
  > Component Unit type

##### `GET` /components/{id}/secondaryData

###### Parameters:

Changed: `id` in `path`

> UUID to identify the component of the company in SiGREEN

###### Return Type:

Changed response : **200 OK**

> A list of secondary data per component

- Changed content type : `application/json`

  - Changed property `items` (array)

  Changed items (object):

  ```text
  * Added property `productionStage` (object)
    > Production stage emissions

      * Property `pcfIncludingBiogenic` (string)
          > Production stage GWP total (incl. bio.).  kg CO2e/declared unit. Either pcfIncludingBiogenic or pcfExcludingBiogenic mandatory.

      * Property `pcfExcludingBiogenic` (string)
          > Production stage GWP total (excl. bio.). kg CO2e/declared unit. Either pcfIncludingBiogenic or pcfExcludingBiogenic mandatory.

      * Property `fossilGhgEmissions` (string)
          > Production stage GWP fossil. kg CO2e/declared unit

      * Property `biogenic` (string)
          > Production stage GWP biogenic. kg CO2e/declared unit

      * Property `dLucGhgEmissions` (string)
          > Production stage GWP direct land use change (dLUC). kg CO2e/declared unit

      * Property `luGhgEmissions` (string)
          > Production stage GWP land use (LU). kg CO2e/declared unit

      * Property `aircraftGhgEmissions` (string)
          > Production stage GWP aircraft emissions. kg CO2e/declared unit

      One of:

  * Added property `distributionStage` (object)
      > Distribution stage emission details

      * Property `pcfIncludingBiogenic` (string)
          > Distribution stage GWP total (incl. bio.). kg CO2e/declared unit. Either pcfIncludingBiogenic or pcfExcludingBiogenic mandatory.

      * Property `pcfExcludingBiogenic` (string)
          > Distribution stage GWP total (excl. bio.). kg CO2e/declared unit. Either pcfIncludingBiogenic or pcfExcludingBiogenic mandatory.

      * Property `biogenic` (string)
          > Distribution stage GWP biogenic. kg CO2e/declared unit

      * Property `fossilGhgEmissions` (string)
          > Distribution stage GWP fossil. kg CO2e/declared unit

      * Property `dLucGhgEmissions` (string)
          > Distribution stage GWP direct land use change (dLUC). kg CO2e/declared unit

      * Property `luGhgEmissions` (string)
          > Distribution stage GWP land use (LU). kg CO2e/declared unit

      * Property `aircraftGhgEmissions` (string)
          > Distribution stage GWP aircraft emissions). kg CO2e/declared unit

  * Deleted property `totalInclBio` (number)
      > Total Incl Bio

  * Deleted property `totalExclBio` (number)
      > total Excl Bio

  * Deleted property `fossil` (string)
      > fossil emissions

  * Deleted property `biogenic` (string)
      > biogenic emissions

  * Deleted property `dLuc` (string)
      > dLuc

  * Deleted property `landUse` (string)
      > land use

  * Deleted property `aircraft` (string)
      > aircraft

  * Changed property `secondaryDataId` (string)
      > UUID to identify the Component SecondaryData of the company in SiGREEN

  * Changed property `quantity` (number -> string)
      > quantity

  ```

##### `POST` /components/{id}/secondaryData

###### Parameters:

Changed: `id` in `path`

> UUID to identify the component of the company in SiGREEN

###### Request:

Changed content type : `application/json`

New required properties:

- `productionStage`

New optional properties:

- `totalExclBio`

- `totalInclBio`

- Added property `productionStage` (object)

  > Production stage emissions

- Added property `distributionStage` (object)

  > Distribution stage emission details

- Deleted property `totalInclBio` (number)

  > total inclusive bio emissions. Either totalInclBio or totalExclBio mandatory.

- Deleted property `totalExclBio` (number)

  > total exclusive bio emissions. Either totalInclBio or totalExclBio mandatory.

- Deleted property `fossil` (string)

  > fossil emissions

- Deleted property `biogenic` (string)

  > biogenic emissions

- Deleted property `dLuc` (string)

  > dLuc

- Deleted property `landUse` (string)

  > land use

- Deleted property `aircraft` (string)

  > aircraft

- Changed property `quantity` (number -> string)

  > quantity

###### Return Type:

Changed response : **201 Created**

> Created

- Changed content type : `application/json`

  - Changed property `secondaryDataId` (string)
    > UUID to identify the component secondaryData of the company in SiGREEN

##### `GET` /components/{id}/secondaryData/{secondaryDataId}

###### Parameters:

Changed: `id` in `path`

> UUID to identify the component of the company in SiGREEN

Changed: `secondaryDataId` in `path`

> UUID to identify the Component Secondary Data of the company in SiGREEN

###### Return Type:

Changed response : **200 OK**

> A list of secondary data per component

- Changed content type : `application/json`

  - Added property `productionStage` (object)

    > Production stage emissions

  - Added property `distributionStage` (object)

    > Distribution stage emission details

  - Deleted property `totalInclBio` (number)

    > Total Incl Bio

  - Deleted property `totalExclBio` (number)

    > total Excl Bio

  - Deleted property `fossil` (string)

    > fossil emissions

  - Deleted property `biogenic` (string)

    > biogenic emissions

  - Deleted property `dLuc` (string)

    > dLuc

  - Deleted property `landUse` (string)

    > land use

  - Deleted property `aircraft` (string)

    > aircraft

  - Changed property `secondaryDataId` (string)

    > UUID to identify the Component SecondaryData of the company in SiGREEN

  - Changed property `quantity` (number -> string)

    > quantity

##### `PUT` /components/{id}/secondaryData/{secondaryDataId}

###### Parameters:

Changed: `id` in `path`

> UUID to identify the component of the company in SiGREEN

Changed: `secondaryDataId` in `path`

> UUID to identify the Component Secondary Data of the company in SiGREEN

###### Request:

Changed content type : `application/json`

New required properties:

- `productionStage`

New optional properties:

- `totalExclBio`

- `totalInclBio`

- Added property `productionStage` (object)

  > Production stage emissions

- Added property `distributionStage` (object)

  > Distribution stage emission details

- Deleted property `totalInclBio` (number)

  > total inclusive bio emissions. Either totalInclBio or totalExclBio mandatory.

- Deleted property `totalExclBio` (number)

  > total exclusive bio emissions. Either totalInclBio or totalExclBio mandatory.

- Deleted property `fossil` (string)

  > fossil emissions

- Deleted property `biogenic` (string)

  > biogenic emissions

- Deleted property `dLuc` (string)

  > dLuc

- Deleted property `landUse` (string)

  > land use

- Deleted property `aircraft` (string)

  > aircraft

- Changed property `quantity` (number -> string)

  > quantity

##### `GET` /components/{id}/linkSuppliers

###### Parameters:

Added: `hasSupplierShare` in `query`

> Filter suppliers who is having supplier share grater than 0. Default would give all results (with and with out share)

Changed: `id` in `path`

> UUID to identify the component of the company in SiGREEN

Changed: `supplierId` in `query`

> UUID to identify the supplier of the company in SiGREEN

###### Return Type:

Changed response : **200 OK**

> A list of secondary data per component

- Changed content type : `application/json`

  - Changed property `items` (array)

  Changed items (object):

  ```text
  * Changed property `componentSupplierLinkId` (string)
    > UUID to identify the link between component and supplier of the company in SiGREEN

  * Changed property `supplierId` (string)
      > UUID to identify the supplier of the company in SiGREEN

  * Changed property `secondaryDataId` (string)
      > UUID to identify the component secondary data of the company in SiGREEN

  ```

##### `POST` /components/{id}/linkSuppliers

###### Parameters:

Changed: `id` in `path`

> UUID to identify the component of the company in SiGREEN

###### Request:

Changed content type : `application/json`

Changed items (object):

- Changed property `supplierId` (string)

  > UUID to identify the supplier of the company in SiGREEN

- Changed property `secondaryDataId` (string)

  > UUID to identify the component secondary data of the company in SiGREEN

###### Return Type:

Changed response : **201 Created**

> Created

- Changed content type : `application/json`

Changed items (object):

```text
* Changed property `componentSupplierLinkId` (string)
  > UUID to identify the link between component and supplier of the company in SiGREEN


* Changed property `supplierId` (string)
    > UUID to identify the supplier of the company in SiGREEN


* Changed property `secondaryDataId` (string)
    > UUID to identify the component secondary data of the company in SiGREEN

```

##### `GET` /components/{id}/linkSuppliers/{componentSupplierLinkId}

###### Parameters:

Changed: `id` in `path`

> UUID to identify the component of the company in SiGREEN

Changed: `componentSupplierLinkId` in `path`

> UUID to identify the linkedSupplier with component of the company in SiGREEN

###### Return Type:

Changed response : **200 OK**

> Get linked suppliers by componentSupplierLinkId with component

- Changed content type : `application/json`

  - Changed property `componentSupplierLinkId` (string)

    > UUID to identify the link between component and supplier of the company in SiGREEN

  - Changed property `supplierId` (string)

    > UUID to identify the supplier of the company in SiGREEN

  - Changed property `secondaryDataId` (string)

    > UUID to identify the component secondary data of the company in SiGREEN

##### `PUT` /components/{id}/linkSuppliers/{componentSupplierLinkId}

###### Parameters:

Changed: `id` in `path`

> UUID to identify the component of the company in SiGREEN

Changed: `componentSupplierLinkId` in `path`

> UUID to identify the linkedSupplier with component of the company in SiGREEN

###### Request:

Changed content type : `application/json`

- Changed property `supplierId` (string)

  > UUID to identify the supplier of the company in SiGREEN

- Changed property `secondaryDataId` (string)

  > UUID to identify the component secondary data of the company in SiGREEN

##### `DELETE` /components/{id}/linkSuppliers/{componentSupplierLinkId}

###### Parameters:

Changed: `id` in `path`

> UUID to identify the component of the company in SiGREEN

Changed: `componentSupplierLinkId` in `path`

> UUID to identify the linkedSupplier with component of the company in SiGREEN

##### `GET` /components/{id}/suppliers/{supplierId}/pcf

###### Parameters:

Changed: `id` in `path`

> UUID to identify the component of the company in SiGREEN

Changed: `supplierId` in `path`

> UUID to identify the supplier of the company in SiGREEN

###### Return Type:

Changed response : **200 OK**

> A component product carbon footprints for component of a supplier.

- Changed content type : `application/json`

  - Changed property `items` (array)

  Changed items (object):

  ```text
  * Added property `productionStage` (object)
      > Production stage emissions

  * Added property `distributionStage` (object)
      > Distribution stage emission details

  * Deleted property `totalPcf` (number)
      > Total PCF

  * Deleted property `dataQuality` (array)
      > Data Quality

  * Changed property `id` (string)
    > UUID to identify the component of the company in SiGREEN

  * Changed property `supplierId` (string)
      > UUID to identify the supplier of the company in SiGREEN

  ```

##### `POST` /components/{id}/suppliers/{supplierId}/pcfRequest

###### Parameters:

Changed: `id` in `path`

> UUID to identify the component of the company in SiGREEN.

Changed: `supplierId` in `path`

> UUID to identify the supplier of the company in SiGREEN

###### Return Type:

Changed response : **201 Created**

> Task is created.

- Changed content type : `application/json`

  - Added property `id` (string)

    > UUID to identify the task of the company in SiGREEN

  - Changed property `taskId` (string)

    > System Generated alphanumeric id to identify the task of the company in SiGREEN

##### `GET` /components/pcfData

###### Parameters:

Added: `componentSupplierLinkId` in `query`

> UUID to identify the linkedSupplier with component of the company in SiGREEN

Added: `accepted` in `query`

> true -> Include PCF's which are received and accepted. false -> Include PCF's which are received. Default all.

Added: `rejected` in `query`

> Include PCF's which are received but rejected

Added: `withdrawn` in `query`

> Include PCF's which are received but withdrawn

Added: `active` in `query`

> Include PCF's which are active

Added: `expired` in `query`

> Include PCF's which are expired

Added: `revoked` in `query`

> Include PCF's which are revoked by Certifier

Added: `specVersion` in `query`

> Spec version or data model standard

Changed: `componentId` in `query`

> UUID to identify the component of the company in SiGREEN

Changed: `supplierId` in `query`

> UUID to identify the supplier of the company in SiGREEN

###### Return Type:

Changed response : **200 OK**

> Get PCF data of components in SiGREEN

- Changed content type : `application/json`

  - Changed property `items` (array)

  Changed items (object):

  ```text
  * Added property `componentSupplierLinkId` (string)
    > UUID to identify the link between component and supplier of the company in SiGREEN

  * Added property `acceptedOn` (string)
      > Proof approved DateTime.DateTime (ISO 8601; UTC Timezone)

  * Added property `pcfId` (string)
      > Id for identifying the pcf data of the company in SiGREEN

  * Added property `accepted` (boolean)
      > PCF's which is received and accepted

  * Added property `rejected` (boolean)
      > PCF is received but rejected

  * Added property `withdrawn` (boolean)
      > PCF which is received but withdrawn

  * Added property `active` (boolean)
      > Whether PCF is active or not

  * Added property `expired` (boolean)
      > PCF which is expired

  * Added property `revoked` (boolean)
      > PCF which is revoked by certifier

  * Deleted property `approvedTimeStamp` (string)
      > Proof approved DateTime.DateTime (ISO 8601; UTC Timezone)

  * Deleted property `dataQuality` (object)
      > Data Quality

  * Changed property `componentId` (string)
      > UUID to identify the component of the company in SiGREEN.

  * Changed property `supplierId` (string)
      > UUID to identify the supplier of the company in SiGREEN.

  * Changed property `pcfData` (object)
      > Verified Data

      * Changed property `pcf` (object)
          > PCF data

          * Changed property `declaredUnit` (string)
              > Component Unit type

  ```

## [1.12.0] - 2024-10-14

### What's New

______________________________________________________________________

#### `POST` /components/{id}/suppliers/{supplierId}/pcfRequest

> Send pcf request to suppliers in SiGREEN.

#### What's Changed

______________________________________________________________________

##### `GET` /components/{id}

###### Parameters

Changed: `id` in `path`

> System generated id to identify the component of the company in SiGREEN.

###### Return Type

Changed response : --200 OK--

> Get component details by id

- Changed content type : `application/json`

  - Changed property `unit` (string)

    > unit types

  - Changed property `identifiers` (array)

    > You have to have one identifier where default parameter is set to true as this represents your main identifier for this entity.

  Changed items (object):

##### `PUT` /components/{id}

###### Parameters

Changed: `id` in `path`

> System generated id to identify the component of the company in SiGREEN.

###### Request

Changed content type : `application/json`

- Changed property `unit` (string)

  > unit types

- Changed property `identifiers` (array)

  > You have to have one identifier where default parameter is set to true as this represents your main identifier for this entity.

Changed items (object):

##### `GET` /components/{id}/secondaryData

###### Parameters

Changed: `id` in `path`

> System generated id to identify the component of the company in SiGREEN.

###### Return Type

Changed response : --200 OK--

> A list of secondary data per component

- Changed content type : `application/json`

  - Changed property `items` (array)

  Changed items (object):

  New required properties:

  - `totalExclBio`
  - `totalInclBio`

  New optional properties:

  - `fossil`

  ```text
  - Added property `createdOn` (string)
    > Created On

  - Added property `componentName` (string)

  - Changed property `secondaryDataId` (string)
    > System generated id to identify the component secondary data of the company in SiGREEN

  ```

##### `POST` /components/{id}/secondaryData

###### Parameters

Changed: `id` in `path`

> System generated id to identify the component of the company in SiGREEN.

###### Request

Changed content type : `application/json`

New required properties

- `totalExclBio`
- `totalInclBio`

New optional properties

- `fossil`

- Added property `createdOn` (string)

  > Created On

###### Return Type

Changed response : --201 Created--

> Created

- Changed content type : `application/json`

  - Changed property `secondaryDataId` (string)
    > System generated id to identify the component secondary data of the company in SiGREEN

##### `GET` /components/{id}/secondaryData/{secondaryDataId}

###### Parameters

Changed: `id` in `path`

> System generated id to identify the component of the company in SiGREEN.

Changed: `secondaryDataId` in `path`

> System generated id to identify the component secondary data of the company in SiGREEN

###### Return Type

Changed response : --200 OK--

> Get component secondary data by secondary data id

- Changed content type : `application/json`

New required properties

- `totalExclBio`
- `totalInclBio`

New optional properties

- `fossil`

```text
- Added property `createdOn` (string)
  > Created On

- Added property `componentName` (string)

- Changed property `secondaryDataId` (string)
  > System generated id to identify the component secondary data of the company in SiGREEN

```

##### `PUT` /components/{id}/secondaryData/{secondaryDataId}

###### Parameters

Changed: `id` in `path`

> System generated id to identify the component of the company in SiGREEN.

Changed: `secondaryDataId` in `path`

> System generated id to identify the component secondary data of the company in SiGREEN

###### Request

Changed content type : `application/json`

New required properties

- `totalExclBio`
- `totalInclBio`

New optional properties

- `fossil`

- Added property `createdOn` (string)

  > Created On

##### `GET` /components/{id}/linkSuppliers

###### Parameters

Changed: `id` in `path`

> System generated id to identify the component of the company in SiGREEN.

Changed: `supplierId` in `query`

> System generated id to identify the supplier of the company in SiGREEN

###### Return Type

Changed response : --200 OK--

> A list of linked suppliers per component

- Changed content type : `application/json`

  - Changed property `items` (array)

  Changed items (object)

  ```text
  - Changed property `componentSupplierLinkId` (string)
    > System generated id to identify the linked supplier with component of the company in SiGREEN

  - Changed property `supplierId` (string)
    > System generated id to identify the supplier of the company in SiGREEN

  - Changed property `secondaryDataId` (string)
    > System generated id to identify the component secondary data of the company in SiGREEN

  ```

##### `POST` /components/{id}/linkSuppliers

###### Parameters

Changed: `id` in `path`

> System generated id to identify the component of the company in SiGREEN.

###### Request

Changed content type : `application/json`

Changed items (object)

- Changed property `supplierId` (string)

  > System generated id to identify the supplier of the company in SiGREEN

- Changed property `secondaryDataId` (string)

  > System generated id to identify the component secondary data of the company in SiGREEN

###### Return Type

Changed response : --201 Created--

> Created

- Changed content type : `application/json`

Changed items (object)

```text
- Added property `componentSupplierLinkId` (string)
  > System generated id to identify the linked supplier with component of the company in SiGREEN

- Changed property `supplierId` (string)
  > System generated id to identify the supplier of the company in SiGREEN

- Changed property `secondaryDataId` (string)
  > System generated id to identify the component secondary data of the company in SiGREEN

```

##### `GET` /components/{id}/linkSuppliers/{componentSupplierLinkId}

###### Parameters

Changed: `id` in `path`

> System generated id to identify the component of the company in SiGREEN.

Changed: `componentSupplierLinkId` in `path`

> System generated id to identify the linked supplier with component of the company in SiGREEN

###### Return Type

Changed response : --200 OK--

> Get linked suppliers by componentSupplierLinkId with component

- Changed content type : `application/json`

  - Changed property `componentSupplierLinkId` (string)

    > System generated id to identify the linked supplier with component of the company in SiGREEN

  - Changed property `supplierId` (string)

    > System generated id to identify the supplier of the company in SiGREEN

  - Changed property `secondaryDataId` (string)

    > System generated id to identify the component secondary data of the company in SiGREEN

##### `PUT` /components/{id}/linkSuppliers/{componentSupplierLinkId}

###### Parameters

Changed: `id` in `path`

> System generated id to identify the component of the company in SiGREEN.

Changed: `componentSupplierLinkId` in `path`

> System generated id to identify the linked supplier with component of the company in SiGREEN

###### Request

Changed content type : `application/json`

New optional properties:

- `componentSupplierLinkId`

- Changed property `supplierId` (string)

  > System generated id to identify the supplier of the company in SiGREEN

- Changed property `secondaryDataId` (string)

  > System generated id to identify the component secondary data of the company in SiGREEN

##### `DELETE` /components/{id}/linkSuppliers/{componentSupplierLinkId}

###### Parameters

Changed: `id` in `path`

> System generated id to identify the component of the company in SiGREEN.

Changed: `componentSupplierLinkId` in `path`

> System generated id to identify the linked supplier with component of the company in SiGREEN

##### `GET` /components/pcf

###### Parameters

Changed: `size` in `query`

> page size

###### Return Type

Changed response : --200 OK--

> A list of component PCF

- Changed content type : `application/json`

  - Changed property `items` (array)

  Changed items (object)

  ```text
  - Changed property `id` (string)
    > System generated id to identify the component of the company in SiGREEN.

  ```

##### `POST` /suppliers

###### Request

Changed content type : `application/json`

- Changed property `identifiers` (array)
  > You have to have one identifier where the default parameter is set to true as this represents your main identifier for this entity.

Changed items (object)

##### `GET` /suppliers

> Get all suppliers for a company in SiGREEN.

###### Parameters

Changed: `from` in `query`

> The start date of the date range when the supplier was created.

Changed: `to` in `query`

> The start date of the date range when the supplier was created.

Changed: `identifier` in `query`

> Supplier Main Identifier to identify the supplier of the company in SiGREEN.

Changed response : --200 OK--

> A list of suppliers

- Changed content type : `application/json`

  - Changed property `items` (array)

  Changed items (object)

  ```text
  - Changed property `id` (string)
    > System generated id to identify the supplier of the company in SiGREEN

  - Changed property `identifiers` (array)
    > You have to have one identifier where the default parameter is set to true as this represents your main identifier for this entity.

    Changed items (object)

  ```

##### `POST` /components

###### Request

Changed content type : `application/json`

- Changed property `unit` (string)

  > unit types

- Changed property `identifiers` (array)

  > You have to have one identifier where default parameter is set to true as this represents your main identifier for this entity.

Changed items (object):

###### Return Type

Changed response : --201 Created--

> Created. Response will return the system id of the created component.

- Changed content type : `application/json`

  - Changed property `id` (string)
    > System Generated Id to identify the component of the company in SiGREEN

##### `GET` /components

###### Parameters

Changed: `identifier` in `query`

> Main Component Identifier to identify the component of the company in SiGREEN

Changed response : --200 OK--

> A list of components

- Changed content type : `application/json`

  - Changed property `items` (array)

  Changed items (object):

  ```text
  - Changed property `id` (string)
    > System generated id to identify the component of the company in SiGREEN.

  - Changed property `unit` (string)
    > unit types

  - Changed property `identifiers` (array)
    > You have to have one identifier where default parameter is set to true as this represents your main identifier for this entity.

    Changed items (object):

  ```

##### `GET` /components/{id}/suppliers/{supplierId}/pcf

###### Parameters

Changed: `id` in `path`

> System generated id to identify the component of the company in SiGREEN.

Changed response : --200 OK--

> A component product carbon footprints for component of a supplier.

- Changed content type : `application/json`

  - Changed property `items` (array)

  Changed items (object):

  ```text
  - Changed property `id` (string)
    > System generated id to identify the component of the company in SiGREEN.

  ```

##### `GET` /components/pcfData

> Get the PCF data for your components in SiGREEN.

###### Parameters

Changed: `componentId` in `query`

> System generated id to identify the component of the company in SiGREEN.

Changed: `pcfId` in `query`

> System Generated Id for identifying the pcf data of the company in SiGREEN

Changed: `from` in `query`

> Proof approved from DateTime. DateTime (ISO 8601; UTC Timezone)

Changed: `to` in `query`

> Proof approved to DateTime. DateTime (ISO 8601; UTC Timezone)

Changed response : --200 OK--

> Get PCF data of components in SiGREEN

- Changed content type : `application/json`

  - Changed property `items` (array)

  Changed items (object)

  ```text
  - Changed property `componentId` (string)
    > System generated id to identify the component of the company in SiGREEN.

  - Changed property `supplierId` (string)
    > System generated id to identify the supplier of the company in SiGREEN

  - Changed property `pcfdata` (object)
    > Verified Data

      - Changed property `pcf` (object)
        > PCF data

          - Changed property `crossSectoralStandardsUsed` (array)
            > List of Cross-sectoral standards applied.

            Removed enum values:

              - `ISO 14067`
              - `Pathfinder v1`
              - `Pathfinder v2`
              - `GHG Protocol Product`
              - `PAS 2050`
              - `ISO 14040-44`
              - `PEF`
              - `Other`
          - Changed property `productOrSectorSpecificRules` (array)

            Removed enum values:

              - `Catena-X Rulebook v1`
              - `Catena-X Rulebook v2`
              - `TFS Guideline v2`
              - `EN 50693`
              - `EN 15804`
              - `BPX 30-323`
              - `Not specified`
          - Changed property `secondaryEmissionFactorSources` (array)
            > Secondary data source & Version.

            Changed items (string):

          - Changed property `assurance` (object)
            > Assurance details

              - Changed property `verificationType` (string)
                > Assurance type.

                Added enum value:

                  - `third party verified PCF`

  ```

## [1.11.0] - 2024-07-22

### What's New

______________________________________________________________________

#### `POST` /suppliers/identifierTypes

> Create a supplier identifier type for a company in SiGREEN.

##### `DELETE` /suppliers/identifierTypes/{id}

> Delete an unused supplier identifier type for a company in SiGREEN.

##### `PATCH` /suppliers/identifierTypes/{id}

> Update supplier identifier type for a company in SiGREEN.

##### `PUT` /suppliers/{id}

> Update a supplier by supplier Id in SiGREEN

##### `POST` /components/identifierTypes

> Create a component identifier type for a company in SiGREEN.

##### `DELETE` /components/identifierTypes/{id}

> Delete an unused component identifier type for a company in SiGREEN.

##### `PATCH` /components/identifierTypes/{id}

> Update a component identifier type for a company in SiGREEN.

##### `PUT` /components/{id}

> Update component details by id

##### `GET` /components/{id}/secondaryData

> Get all component secondary data for component in SiGREEN

##### `POST` /components/{id}/secondaryData

> Create secondary data with component.

##### `GET` /components/{id}/secondaryData/{secondaryDataId}

> Get component secondary data by secondary data id

##### `PUT` /components/{id}/secondaryData/{secondaryDataId}

> Update component secondary data by secondary data id.

##### `GET` /components/{id}/linkSuppliers

> Get all linked suppliers with component

##### `POST` /components/{id}/linkSuppliers

> Link suppliers and secondary data with component

##### `GET` /components/{id}/linkSuppliers/{componentSupplierLinkId}

> Get linked suppliers by componentSupplierLinkId with component

##### `PUT` /components/{id}/linkSuppliers/{componentSupplierLinkId}

> Update location/share/secondary data/supplier product id by component supplier id

##### `DELETE` /components/{id}/linkSuppliers/{componentSupplierLinkId}

> Delete link between component and supplier

##### `GET` /components/{id}/suppliers/{supplierId}/pcf

> Get an export of a PCF of a component without cryptographic material. If you need cryptographic verifiability of this information please use DIDComm APIs.

#### What's Changed

______________________________________________________________________

##### `GET` /components/pcf

> Get an aggregated export of a PCF of a component without cryptographic material. If you need cryptographic verifiability of this information please use DIDComm APIs.

###### Parameters

Added: `componentIds` in `query`

> You can specify more than one componentId(System generated id) here (separated by comma).

Deleted: `supplierId` in `query`

Deleted: `componentId` in `query`

Changed: `page` in `query`

> page number

Changed: `size` in `query`

> page size

###### Return Type

New response : --403 Forbidden--

> Forbidden. Deleted response : --404 Not Found-- Not found. Changed response : --200 OK-- A list of Component PCF

- Changed content type : `application/json`

  - Added property `items` (array)

  Items (object):

  ```text
  - Property `id` (string)
    > System generated id to identify the component of the company in SiGREEN

  - Property `totalPcf` (number)
    > Total PCF

  - Property `pcfUnit` (string)
    > PCF Unit Types

    Enum values:

      - `kgCO2e/piece`
      - `kgCO2e/kg`
      - `kgCO2e/m2`
      - `kgCO2e/m3`
      - `kgCO2e/kWh`
      - `kgCO2e/tkm`
      - `kgCO2e/MJ`
      - `kgCO2e/L`

  ```

  - Deleted property `results` (array)

  - Changed property `hasNext` (boolean)

    > true if next page exists (page < totalPages)

  - Changed property `hasPrev` (boolean)

    > true if a previous page exists (page > 0)

  - Changed property `page` (number)

    > The current page-number (starts with 0)

  - Changed property `pageNumber` (number)

    > The current page-number (starts with 1)

  - Changed property `size` (number)

    > The size of elements in the response

  - Changed property `totalPages` (number)

    > The total available pages

Changed response : --400 Bad Request--

> Bad request.

- New content type : `application/problem+json`
- Deleted content type : `application/json`

Changed response : --401 Unauthorized--

> Unauthorized.

- New content type : `application/problem+json`

Changed response : --500 Internal Server Error--

> Unexpected error.

- New content type : `application/problem+json`

##### `GET` /components/pcfData

###### Parameters

Changed: `componentId` in `query`

> System generated id to identify the component of the company in SiGREEN

Changed: `supplierId` in `query`

> System generated id to identify the supplier of the company in SiGREEN

Changed: `page` in `query`

> Page number

Changed: `size` in `query`

> Page size

###### Return Type

New response : --401 Unauthorized--

> Unauthorized. New response : --403 Forbidden-- Forbidden. Changed response : --200 OK-- Get PCF data of components in SiGREEN

- Changed content type : `application/json`

  - Added property `items` (array)

  Items (object):

  ```text
  - Property `componentIdentifier` (string)
    > Component Main Identifier.

  - Property `componentName` (string)
    > component Name.

  - Property `componentId` (string)
    > System generated id to identify the component of the company in SiGREEN.

  - Property `supplierId` (string)
    > System generated id to identify the supplier of the company in SiGREEN.

  - Property `supplierIdentifier` (string)
    > Supplier Main Identifier.

  - Property `supplierName` (string)
    > Supplier name.

  - Property `supplierProductId` (string)
    > supplier product id

  - Property `location` (string)
    > Supplier Location.

  - Property `approvedTimeStamp` (string)
    > Proof approved DateTime.DateTime (ISO 8601; UTC Timezone)

  - Property `pcfState` (string)
    > State of the pcf to say if it self certified or certified by a 3rd party certifier

  - Property `dataQuality` (object)
    > Data Quality

      - Property `typeOfQuality` (string)
        > Type of quality

        Enum values:

          - `Transparency`
          - `Reliability`
          - `Validity & Data Accuracy`
      - Property `percentage` (number)
        > Data quality calculated in percentage

  - Property `pcfData` (object)
    > Verified Data

      - Property `company` (object)
        > Company Details

          - Property `companyName` (string)
            > Company name.

          - Property `companyIds` (array)
            > Company IDs.

            Items (string):

          - Property `companyProducer` (string)
            > Name of producer, if it differs from the supplier of product.

          - Property `companyContactEmail` (string)
            > Contact email address.

          - Property `companyContactPhone` (string)
            > Contact phone.

      - Property `product` (object)
        > Product details

          - Property `name` (string)
            > name of product

          - Property `family` (string)
            > name of product family

          - Property `identifiers` (array)
            > You have to have one identifier where default parameter is set to true as this represents your main identifier for this entity.

            Items (object):

              - Property `idType` (string)
                > identifier type

              - Property `value` (string)
                > identifier value

              - Property `default` (boolean)
                > whether main identifier type or not

          - Property `description` (string)
            > product description

          - Property `weight` (string)
            > weight of product

          - Property `quantity` (string)
            > product quantity

          - Property `unitType` (string)
            > unit types

            Enum values:

              - `piece`
              - `kg`
              - `m2`
              - `m3`
              - `kWh`
              - `km`
              - `TEU`
              - `pkm`
              - `tkm`
              - `MJ`
              - `L`
          - Property `location` (string)
            > Product location

          - Property `factory` (array)
            > Factory

            Items (string):

      - Property `pcf` (object)
        > PCF data

          - Property `scopePcf` (object)
            > Scope of PCF

              - Property `specVersion` (array)
                > Data model and version.

                Items (string):

              - Property `limitedOrFullDec` (string)
                > Limited or full declaration.

                Enum values:

                  - `Mandatory inputs only`
                  - `Mandatory and optional inputs`
              - Property `partialFullPcf` (string)
                > Partial or a full PCF.

                Enum values:

                  - `Cradle-to-gate`
                  - `Cradle-to-grave`
              - Property `productionStage` (string)
                > Scope Pcf Type

                Enum values:

                  - `included`
                  - `excluded`
              - Property `materialAcquisitionAndPreProcessingSubStage` (string)
                > Scope Pcf Type

                Enum values:

                  - `included`
                  - `excluded`
              - Property `rawMaterialAcquistionSub2Stage` (string)
                > Scope Pcf Type

                Enum values:

                  - `included`
                  - `excluded`
              - Property `preprocessingSub2Stage` (string)
                > Scope Pcf Type

                Enum values:

                  - `included`
                  - `excluded`
              - Property `transportToManufacturingSiteSub2Stage` (string)
                > Scope Pcf Type

                Enum values:

                  - `included`
                  - `excluded`
              - Property `manufacturingSubStage` (string)
                > Scope Pcf Type

                Enum values:

                  - `included`
                  - `excluded`
              - Property `assemblySub2Stage` (string)
                > Scope Pcf Type

                Enum values:

                  - `included`
                  - `excluded`
              - Property `packagingEmissionsIncluded` (string)
                > Scope Pcf Type

                Enum values:

                  - `included`
                  - `excluded`
              - Property `eolTreatmentOfGeneratedWasteMfgSub2Stage` (string)
                > Scope Pcf Type

                Enum values:

                  - `included`
                  - `excluded`
              - Property `distributionStage` (string)
                > Scope Pcf Type

                Enum values:

                  - `included`
                  - `excluded`
              - Property `transportAToDistributerSubStage` (string)
                > Scope Pcf Type

                Enum values:

                  - `included`
                  - `excluded`
              - Property `reconditioningAtDistributorSubStage` (string)
                > Scope Pcf Type

                Enum values:

                  - `included`
                  - `excluded`
              - Property `transportBToPlaceOfUseSubStage` (string)
                > Scope Pcf Type

                Enum values:

                  - `included`
                  - `excluded`
              - Property `installationAndAssemblyStage` (string)
                > Scope Pcf Type

                Enum values:

                  - `included`
                  - `excluded`
              - Property `installationSubStage` (string)
                > Scope Pcf Type

                Enum values:

                  - `included`
                  - `excluded`
              - Property `eolTreatmentOfGeneratedWasteInstallationSubStage` (string)
                > Scope Pcf Type

                Enum values:

                  - `included`
                  - `excluded`
              - Property `useStage` (string)
                > Scope Pcf Type

                Enum values:

                  - `included`
                  - `excluded`
              - Property `useSubStage` (string)
                > Scope Pcf Type

                Enum values:

                  - `included`
                  - `excluded`
              - Property `maintenanceSubStage` (string)
                > Scope Pcf Type

                Enum values:

                  - `included`
                  - `excluded`
              - Property `repairSubStage` (string)
                > Scope Pcf Type

                Enum values:

                  - `included`
                  - `excluded`
              - Property `replacementSubStage` (string)
                > Scope Pcf Type

                Enum values:

                  - `included`
                  - `excluded`
              - Property `eolTreatmentOfGeneratedWasteUseSubStage` (string)
                > Scope Pcf Type

                Enum values:

                  - `included`
                  - `excluded`
              - Property `refurbishmentSubStage` (string)
                > Scope Pcf Type

                Enum values:

                  - `included`
                  - `excluded`
              - Property `operationalEnergyUseSubStage` (string)
                > Scope Pcf Type

                Enum values:

                  - `included`
                  - `excluded`
              - Property `operationalWaterUseSubStage` (string)
                > Scope Pcf Type

                Enum values:

                  - `included`
                  - `excluded`
              - Property `eolStage` (string)
                > Scope Pcf Type

                Enum values:

                  - `included`
                  - `excluded`
              - Property `demolitionSubStage` (string)
                > Scope Pcf Type

                Enum values:

                  - `included`
                  - `excluded`
              - Property `transportSubStage` (string)
                > Scope Pcf Type

                Enum values:

                  - `included`
                  - `excluded`
              - Property `wasteProcessingSubStage` (string)
                > Scope Pcf Type

                Enum values:

                  - `included`
                  - `excluded`
              - Property `disposalSubStage` (string)
                > Scope Pcf Type

                Enum values:

                  - `included`
                  - `excluded`
              - Property `reuseRecoveryRecyclingPotentialStage` (string)
                > Scope Pcf Type

                Enum values:

                  - `included`
                  - `excluded`
          - Property `declaredUnit` (string)
            > unit types

            Enum values:

              - `piece`
              - `kg`
              - `m2`
              - `m3`
              - `kWh`
              - `km`
              - `TEU`
              - `pkm`
              - `tkm`
              - `MJ`
              - `L`
          - Property `unitaryProductAmount` (string)
            > Quantity (of declared unit)

          - Property `productMassPerDeclaredUnit` (string)
            > Product mass [kg] per declared unit.

          - Property `id` (string)
            > PCF ID.Id will represent like UUID v4.

          - Property `precedingPfIds` (array)
            > Previous PCF IDs.Id will represent like UUID v4.

            Items (string):

          - Property `version` (string)
            > PCF version, the value can be in decimals.

          - Property `pfStatus` (string)
            > PCF status

            Enum values:

              - `Active`
              - `Deprecated`
          - Property `pfStatusComment` (string)
            > PCF status comment

          - Property `exemptedEmissionsPercent` (string)
            > Cut-off rule %

          - Property `exemptedEmissionsDescription` (string)
            > Cut-off and exemption rules explanation

          - Property `boundaryProcessesDescription` (string)
            > Important unit processes and used technologies

          - Property `typeRecycledContent` (string)
            > Type of recycled content

            Enum values:

              - `post-industrial`
              - `post-consumer`
          - Property `ccuCo2Origin` (string)
            > CCU CO2-origin

          - Property `geographyCountrySubdivision` (string)
            > City/state as country subdivision.according to ISO 3166-2 alpha-2 subdivision code

          - Property `geographyCountry` (string)
            > Geography country.according to ISO 3166-2 alpha-2 country code

          - Property `geographyRegionOrSubregion` (string)
            > Geography with region or subregion

            Enum values:

              - `Africa`
              - `Americas`
              - `Asia`
              - `Europe`
              - `Oceania`
              - `Australia and New Zealand`
              - `Central Asia`
              - `Eastern Asia`
              - `Eastern Europe`
              - `Latin America and the Caribbean`
              - `Melanesia`
              - `Micronesia`
              - `Northern Africa`
              - `Northern America`
              - `Northern Europe`
              - `Polynesia`
              - `South-eastern Asia`
              - `Southern Asia`
              - `Southern Europe`
              - `Sub-Saharan Africa`
              - `Western Asia`
              - `Western Europe`
              - `Global`
          - Property `locationProductHandover` (string)
            > Location of product handover

            Enum values:

              - `Own factory gate`
              - `Customer gate`
              - `Other`
          - Property `referencePeriodStart` (string)
            > Reference period start.DateTime (ISO 8601; UTC Timezone)

          - Property `referencePeriodEnd` (string)
            > Reference period end.DateTime (ISO 8601; UTC Timezone)

          - Property `created` (string)
            > Date of issue.DateTime (ISO 8601; UTC Timezone)

          - Property `updated` (string)
            > Date update.DateTime (ISO 8601; UTC Timezone)

          - Property `validityPeriodStart` (string)
            > Validity period start.DateTime (ISO 8601; UTC Timezone)

          - Property `validityPeriodEnd` (string)
            > Validity period end. DateTime (ISO 8601; UTC Timezone)

          - Property `crossSectoralStandardsUsed` (array)
            > Cross-sectoral standards applied.

            Items (string):

            Enum values:

              - `ISO 14067`
              - `Pathfinder v1`
              - `Pathfinder v2`
              - `GHG Protocol Product`
              - `PAS 2050`
              - `ISO 14040-44`
              - `PEF`
              - `Other`
          - Property `productOrSectorSpecificRules` (array)
            > Product or sector rules applied.

            Items (string):

            Enum values:

              - `Catena-X Rulebook v1`
              - `Catena-X Rulebook v2`
              - `TFS Guideline v2`
              - `EN 50693`
              - `EN 15804`
              - `BPX 30-323`
              - `Not specified`
          - Property `operator` (string)
            > select "other".

            Enum values:

              - `PEF`
              - `EPD International`
              - `Other`
          - Property `otherOperatorName` (string)
            > Publisher's URN of product or sector specific guidance.

          - Property `biogenicAccountingMethodology` (string)
            > Methodology used to account for biogenic emissions and removals applied.

            Enum values:

              - `PEF`
              - `ISO`
              - `GHGP`
              - `Quantis`
          - Property `characterizationFactorsTimeHorizon` (string)
            > Time horizon of GWP values.

            Enum values:

              - `GWP100`
              - `GWP20`
              - `GWP500`
              - `GWP1000`
              - `Unspecified`
          - Property `characterizationFactors` (string)
            > IPCC report version of GWP values.

            Enum values:

              - `AR1`
              - `AR2`
              - `AR3`
              - `AR4`
              - `AR5`
              - `AR6`
              - `unspecified`
          - Property `characterizationFactorsClimateCarbonFeedback` (string)
            > Are climate-carbon feedbacks included in the GWP?

            Enum values:

              - `yes`
              - `no`
              - `partly`
              - `unspecified`
          - Property `characterizationFactorsClimateCarbonFeedbackLciaMethod` (string)
            > LCIA method applied in assessment.

            Enum values:

              - `Greenhouse Gas Protocol`
              - `PEF`
              - `IPCC 2013 GWP100a`
              - `IPCC 2013 GWP20a`
              - `IPCC 2007 GWP100a`
              - `IPCC 2007 GWP20a`
              - `IPCC 2001 GWP100a`
              - `IPCC 2001 GWP20a`
              - `ReCiPe 2016 Midpoint (E)`
              - `ReCiPe 2016 Midpoint (I)`
              - `ReCiPe 2016 Midpoint (H)`
              - `CML-IA baseline`
              - `CML 1992`
              - `CML 2 baseline 2000`
              - `CML 2001 (all impact categories)`
              - `EDIP 2003`
              - `EPD (2008), EPD (2013)`
              - `ILCD 2011 Midpoint+`
              - `IMPACT 2002+`
              - `TRACI 2.1`
              - `BEES+`
              - `other`
              - `unspecified`
          - Property `characterizationFactorsGhgExclusions` (string)
            > Exclusion of greenhouse gases.

          - Property `allocationRulesDescription` (string)
            > Allocation rules used.

          - Property `recycledMaterialAllocationApproach` (string)
            > Recycled material allocation approach.

            Enum values:

              - `Recycled content method`
              - `Closed loop approximation method`
          - Property `allocationWasteIncineration` (string)
            > Allocation approach used for waste incineration with energy recovery.

            Enum values:

              - `cut-off`
              - `reverse cut-off`
              - `system expansion`
          - Property `allocationRecycledCarbon` (string)
            > Allocation approach used for recycled material & c-content.

            Enum values:

              - `upstream system expansion`
              - `cut-off`
          - Property `allocationCcuCarbon` (string)
            > Allocation approach used for CCU.

          - Property `primaryDataShare` (string)
            > Primary Data Share (PDS).

          - Property `sourceOfPrimaryData` (string)
            > Sources for primary data.

          - Property `secondaryEmissionFactorSources` (array)
            > Secondary data source & Version.

            Items (string):

          - Property `secondaryEmissionFactorSourcesVersion` (string)
            > Secondary data source version.

  ```

  - Deleted property `results` (array)

  - Changed property `hasNext` (boolean)

    > true if next page exists (page < totalPages)

  - Changed property `hasPrev` (boolean)

    > true if a previous page exists (page > 0)

  - Changed property `page` (number)

    > The current page-number (starts with 0)

  - Changed property `pageNumber` (number)

    > The current page-number (starts with 1)

  - Changed property `size` (number)

    > The size of elements in the response

  - Changed property `totalPages` (number)

    > The total available pages

Changed response : --400 Bad Request--

> Bad request.

- New content type : `application/problem+json`
- Deleted content type : `application/json`

Changed response : --500 Internal Server Error--

> Unexpected error.

- New content type : `application/problem+json`

##### `GET` /components/identifierTypes

> Get component identifier types for a company in SiGREEN.

###### Parameters

Added: `page` in `query`

> Page Number, usually starts with 0

Added: `size` in `query`

> Size of the page default value is 10

###### Return Type

New response : --403 Forbidden--

> Forbidden. Deleted response : --400 Bad Request-- Bad request. Changed response : --401 Unauthorized-- Unauthorized.

- New content type : `application/problem+json`

Changed response : --500 Internal Server Error--

> Unexpected error.

- New content type : `application/problem+json`

Changed response : --200 OK--

> List of component identifier types.

- Changed content type : `application/json`

  - Changed property `hasNext` (boolean)

    > true if next page exists (page < totalPages)

  - Changed property `hasPrev` (boolean)

    > true if a previous page exists (page > 0)

  - Changed property `page` (number)

    > The current page-number (starts with 0)

  - Changed property `pageNumber` (number)

    > The current page-number (starts with 1)

  - Changed property `size` (number)

    > The size of elements in the response

  - Changed property `totalPages` (number)

    > The total available pages

  - Changed property `identifierTypes` (array)

  Changed items (object):

  New required properties -`id` -`idType` -`isMainIdentifierType`

  ```text
  - Added property `id` (string)
    > A unique id for identifier type.

  - Changed property `idType` (string)
    > identifier type value.

  - Changed property `isMainIdentifierType` (boolean)
    > Whether it is main identifier type or not.

  ```

##### `GET` /suppliers/identifierTypes

> Get supplier identifier types for a company in SiGREEN.

###### Parameters

Added: `page` in `query`

> Page Number, usually starts with 0

Added: `size` in `query`

> Size of the page default value is 10

###### Return Type

New response : --403 Forbidden--

> Forbidden. Deleted response : --400 Bad Request-- Bad request. Changed response : --401 Unauthorized-- Unauthorized.

- New content type : `application/problem+json`

Changed response : --500 Internal Server Error--

> Unexpected error.

- New content type : `application/problem+json`

Changed response : --200 OK--

> List of supplier identifier types

- Changed content type : `application/json`

  - Changed property `hasNext` (boolean)

    > true if next page exists (page < totalPages)

  - Changed property `hasPrev` (boolean)

    > true if a previous page exists (page > 0)

  - Changed property `page` (number)

    > The current page-number (starts with 0)

  - Changed property `pageNumber` (number)

    > The current page-number (starts with 1)

  - Changed property `size` (number)

    > The size of elements in the response

  - Changed property `totalPages` (number)

    > The total available pages

  - Changed property `identifierTypes` (array)

  Changed items (object):

  New required properties: -`id`

  - `idType`
  - `isMainIdentifierType`

  ```text
  - Added property `id` (string)
    > A unique id for identifier type.

  - Changed property `idType` (string)
    > identifier type value.

  - Changed property `isMainIdentifierType` (boolean)
    > Whether it is main identifier type or not.

  ```

# Quick Start

For getting started with our Application and our APIs, kindly refer to [Getting Started](../getting-started.html).

For general information about the **Procurement** functionality, refer to the respective section in the SiGREEN Help: [Procurement made easy](https://siemens-help.stonly.com/kb/en/procurement-made-easy-183582).

## Concepts and Glossary

For detailed information about the concepts and terminology, please refer to [Concepts and Glossary](../concepts-glossary.html).

## Procurement API

The Procurement API, part of SiGREEN's suite of APIs, offers a streamlined solution for managing component-level emissions within procurement processes. With various endpoints mentioned below, users can efficiently access and analyze crucial data related to components and suppliers.

## Suppliers

**Prerequisite**: In order to create a supplier, it is mandatory to create the supplier identifier type first.

Getting started with the **Procurement** APIs involves the following steps with respect to the **Suppliers**:

1. Create a supplier identifier type for a company in SiGREEN. The identifier type created initially in the beginning acts as the main identifier type.

   Example Request

   ```bash
     curl --location 'https://app.sigreen.siemens.com/api/suppliers/identifierTypes' \
     --header 'Content-Type: application/json' \
     --header 'Authorization: Bearer TOKEN' \
     --data '{
         "idType": "Supplier ID"
     }'

   ```

   Example Response

   ```bash
     {
         "id": "0190832b-b69b-778f-a843-a2a787a0827d"
     }

   ```

1. Get supplier identifier types for a company in SiGREEN.

   Example Request

   ```bash
     curl --location 'https://app.sigreen.siemens.com/api/suppliers/identifierTypes' \
     --header 'Authorization: Bearer TOKEN' \

   ```

   Example Response

   ```bash
     {
         "items": [

             {
                 "id": "0190832b-b69b-778f-a843-a2a787a0827d",
                 "idType": "Supplier ID",
                 "isMainIdentifierType": true
             }
         ],
         "hasNext": false,
         "hasPrev": false,
         "page": 0,
         "pageNumber": 1,
         "size": 1,
         "totalPages": 1
     }

   ```

1. Create a supplier in SiGREEN.

   Example Request

   ```bash
     curl --location 'https://app.sigreen.siemens.com/api/suppliers' \
     --header 'Content-Type: application/json' \
     --header 'Authorization: Bearer TOKEN' \
     --data-raw '{
         "name": "Supplier Company Limited",
         "identifiers": [
             {
                 "idType": "Supplier ID",
                 "value": "SUP-123",
                 "default": true
             }
         ],
         "email": "demo@demo.com",
         "contactNumber": "+91 234-234-2445",
         "website": "www.demo.com",
         "locations": [
             "12459 Berlin, Siemensstraße 42"
         ],
         "comment": "Proc Supplier"
     }'

   ```

   Example Response

   ```bash
     {
         "id": "01908348-1ade-718b-9895-0b53c9e4a4f9"
     }

   ```

1. Get supplier details by ID for a company in SiGREEN.

   Example Request

   ```bash
     curl --location 'https://app.sigreen.siemens.com/api/suppliers/01908348-1ade-718b-9895-0b53c9e4a4f9' \
     --header 'Authorization: Bearer TOKEN' \
     --data ''

   ```

   Example Response

   ```bash
     {
         "id": "01908348-1ade-718b-9895-0b53c9e4a4f9",
         "name": "Supplier Company Limited",
         "identifiers": [
             {
                 "idType": "Supplier ID",
                 "value": "SUP-123",
                 "default": true
             }
         ],
         "email": "demo@demo.com",
         "contactNumber": "+49 911 2155- 4444",
         "website": "www.demo.com",
         "locations": [
         "12459 Berlin, Siemensstraße 42"
         ],
         "comment": "Proc Supplier",
         "status": "ACTIVE",
         "createdOn": "2023-01-01T12:00:00",
         "updatedOn": "2023-01-02T12:00:00"
     }

   ```

note

Please note that this change will only affect new data. Existing data will not be affected unless they are updated.

With the other available Supplier APIs, you can perform the following operations:

1. Update supplier identifier type for a company in SiGREEN.
1. Update a supplier by supplier ID in SiGREEN.
1. Delete a supplier by supplier ID in SiGREEN.
1. Invite a supplier to SiGREEN.
1. Cancel a supplier invite to SiGREEN.
1. Get all suppliers for one company in SiGREEN.
1. Delete an unused supplier identifier type for a company in SiGREEN.

For more information in detail about the **Parameters**, **Request Body**, and **Responses** refer to the [API Reference](api-procurement.html) under Procurement API(s).

## Components

**Prerequisite**: In order to create a component, it is mandatory to create the component identifier type first.

Getting started with the Procurement APIs involves the following steps with respect to the **Components**:

1. Create a component identifier type for a company in SiGREEN. The identifier type created initially in the beginning acts as the main identifier type.

   Example Request

   ```bash
     curl --location 'https://app.sigreen.siemens.com/api/components/identifierTypes' \
     --header 'Content-Type: application/json' \
     --header 'Authorization: Bearer TOKEN' \
     --data '{
         "idType": "SKU"
     }'

   ```

   Example Response

   ```bash
     {
         "id": "0190835f-f167-7a6c-b8ab-2d5cb2c1d7e8"
     }

   ```

1. Get component identifier types for a company in SiGREEN.

   Example Request

   ```bash
     curl --location 'https://app.sigreen.siemens.com/api/components/identifierTypes?size=1' \
     --header 'Authorization: Bearer TOKEN' \

   ```

   Example Response

   ```bash
     {
         "items": [
             {
                 "id": "0190835f-f167-7a6c-b8ab-2d5cb2c1d7e8",
                 "idType": "SKU",
                 "isMainIdentifierType": true
             }
         ],
         "hasNext": false,
         "hasPrev": false,
         "page": 0,
         "pageNumber": 1,
         "size": 1,
         "totalPages": 1
     }

   ```

1. Create a component in SiGREEN.

   Example Request

   ```bash
     curl --location 'https://app.sigreen.siemens.com/api/components' \
     --header 'Content-Type: application/json' \
     --header 'Authorization: Bearer TOKEN' \
     --data '{
         "name": "Frames",
         "materialCategory": "Electronics",
         "unit": "piece",
         "weight": "3.5",
         "identifiers": [
             {
                 "idType": "SKU",
                 "value": "FRM1234",
                 "default": true
             }
         ]
     }'

   ```

   Example Response

   ```bash
     {
         "id": "0190838b-4469-76fc-b4bc-414c71526f2f"
     }

   ```

1. Get component details by ID.

   Example Request

   ```bash
     curl --location --request GET 'https://app.sigreen.siemens.com/api/components/0190838b-4469-76fc-b4bc-414c71526f2f' \
     --header 'Content-Type: application/json' \
     --header 'Authorization: Bearer TOKEN' \
     --data ''

   ```

   Example Response

   ```bash
     {
         "id": "0190838b-4469-76fc-b4bc-414c71526f2f",
         "name": "Frames",
         "materialCategory": "Electronics",
         "unit": "piece",
         "weight": "3.5",
         "identifiers": [
             {
                 "idType": "SKU",
                 "value": "FRM1234",
                 "default": true
             },
         ],
         "createdOn": "2024-04-19T00:00:00.000Z",
         "updatedOn": "2024-04-19T00:00:00.000Z"
     }

   ```

1. Create secondary data with component. Either totalInclBio or totalExclBio mandatory.

   Example Request

   ```bash
     curl --location 'https://app.sigreen.siemens.com/api/components/0190838b-4469-76fc-b4bc-414c71526f2f/secondaryData' \
     --header 'Content-Type: application/json' \
     --header 'Authorization: Bearer TOKEN' \
     --data-raw '{
         "email": "abc@testing.com",
         "productionStage": {
         "pcfIncludingBiogenic": "79.86801562912491",
         "pcfExcludingBiogenic": "79.86801562912432",
         "fossilGhgEmissions": "79.86801562912497",
         "biogenic": "79.86801562912497",
         "luGhgEmissions": "79.86801562912497",
         "aircraftGhgEmissions": "79.86801562912497",
         "dLucGhgEmissions": "79.86801562912497"
       },
       "distributionStage": {
         "pcfIncludingBiogenic": "79.86801562912494",
         "pcfExcludingBiogenic": "79.8680156291287",
         "fossilGhgEmissions": "79.86801562912497",
         "biogenic": "79.86801562912497",
         "luGhgEmissions": "79.86801562912497",
         "aircraftGhgEmissions": "79.86801562912497",
         "dLucGhgEmissions": "79.86801562912497"
       },
         "quantity": 1,
         "validityPeriodStart": "2024-06-04T08:30:00.000Z",
         "region": "Europe",
         "comment": "No Comments",
         "dataSource": "Database",
         "dataVerification": "Proxy data"
     }'

   ```

   Example Response

   ```bash
     {
         "secondaryDataId": "0190839e-1255-753b-8c52-a3202978a983"
     }

   ```

1. Get component secondary data by secondary data ID.

   Example Request

   ```bash
     curl --location 'https://app.sigreen.siemens.com/api/components/0190838b-4469-76fc-b4bc-414c71526f2f/secondaryData/0190839e-1255-753b-8c52-a3202978a983' \
     --header 'Authorization: Bearer TOKEN' \
     --data ''

   ```

   Example Response

   ```bash
     {
         "quantity": "1.0",
         "dataSource": "Database",
         "dataVerification": "Proxy data",
         "region": "Europe",
         "validityPeriodStart": "2024-06-04 08:30:00.0",
         "comment": "No Comments",
         "secondaryDataId": "0190839e-1255-753b-8c52-a3202978a983",
         "email": "abc@testing.com"
         "productionStage": {
         "pcfIncludingBiogenic": "79.86801562912491",
         "pcfExcludingBiogenic": "79.86801562912432",
         "fossilGhgEmissions": "79.86801562912497",
         "biogenic": "79.86801562912497",
         "luGhgEmissions": "79.86801562912497",
         "aircraftGhgEmissions": "79.86801562912497",
         "dLucGhgEmissions": "79.86801562912497"
       },
       "distributionStage": {
         "pcfIncludingBiogenic": "79.86801562912494",
         "pcfExcludingBiogenic": "79.8680156291287",
         "fossilGhgEmissions": "79.86801562912497",
         "biogenic": "79.86801562912497",
         "luGhgEmissions": "79.86801562912497",
         "aircraftGhgEmissions": "79.86801562912497",
         "dLucGhgEmissions": "79.86801562912497"
       },
     }

   ```

1. Link suppliers and secondary data with component.

   Example Request

   ```bash
     curl --location 'https://app.sigreen.siemens.com/api/components/0190838b-4469-76fc-b4bc-414c71526f2f/linkSuppliers' \
     --header 'Content-Type: application/json' \
     --header 'Authorization: Bearer TOKEN' \
     --data '[
         {
             "supplierId": "01908348-1ade-718b-9895-0b53c9e4a4f9",
             "supplierLocation": "12459 Berlin, Siemensstraße 42",
             "share": 100,
             "supplierProductId": "SUPCOMP1",
             "secondaryDataId": "018fe8a5-349f-7e94-8032-d57f9420e2a4"
         }
     ]'

   ```

   Example Response

   ```bash
     [
         {
             "supplierLocation": "12459 Berlin, Siemensstraße 42",
             "share": "100.0",
             "supplierProductId": "SUPCOMP1",
             "supplierId": "01908348-1ade-718b-9895-0b53c9e4a4f9",
             "secondaryDataId": "018fe8a5-349f-7e94-8032-d57f9420e2a4"
         }
     ]

   ```

1. Send pcf request to suppliers in SiGREEN.

   The PCF Request API allows you to send a Product Carbon Footprint (PCF) request to suppliers within your supply chain, helping track and report the carbon footprint of specific components. This is essential for maintaining transparency and sustainability in the supply chain.

   This API sends a request to the supplier to provide the carbon footprint information for a specific component. The request is based on the unique identifiers for both the component and the supplier.

   On a successful request, the supplier is notified via email and also via the standard SiGREEN notification in the application. The supplier will then be responsible for responding with the PCF information.

   Example Request

   ```bash
     curl --location 'https://app.sigreen.siemens.com/api/components/019228f3-8d22-73f7-a8a9-673996b71326/suppliers/cb688ef3-a61f-455f-b308-4c80f87e8db6/pcfRequest' \
     --header 'Content-Type: application/json' \
     --header 'Authorization: Bearer TOKEN' \
     --data '{
         "pcfStandard": "TFS V2",
         "location": "Mumbai",
         "message": "PCF request message",
         "identifiers": [
             {
                 "idType": "ean",
                 "value": "4667"
             }
         ]
     }'

   ```

   Example Response

   ```bash
      {
         "taskId": "T0247",                
      }

   ```

With the other available Component APIs, you can perform the following operations:

1. Update a component identifier type for a company in SiGREEN.
1. Get all components for one company in SiGREEN.
1. Delete a component by ID for a company in SiGREEN.
1. Get all component secondary data for component in SiGREEN.
1. Update component secondary data by secondary data ID.
1. Get all linked suppliers with component.
1. Get linked suppliers by componentSupplierLinkId with component.
1. Update location/share/secondary data/supplier product ID by component supplier ID.
1. Delete an unused component identifier type for a company in SiGREEN.
1. Delete link between component and supplier. Deletion of records supports only soft delete.

For more information in detail about the **Parameters**, **Request Body**, and **Responses** refer to the [API Reference](api-procurement.html) under Procurement API(s).

## Components PCF

With the available Components PCF APIs, you can perform the following operations:

1. Get an aggregated export of a PCF of a component.

1. Get an export of a PCF of a component.

1. Get the average emissions for components across your company.

1. GET the PCF data for your components in SiGREEN.

   With this API, you can retrieve the PCF data for your components shared by the supplier in different scenarios:

   You can retrieve the PCF data for all the components available in SiGREEN for your respective company.

   Example Request

   ```bash
     curl --location 'https://app.sigreen.siemens.com/api/components/pcfData' \
     --header 'Content-Type: application/json' \
     --header 'Authorization: Bearer TOKEN' \

   ```

   You can retrieve the PCF data shared by the supplier in a specific time range.

   Example Request

   ```bash
     curl --location 'https://app.sigreen.siemens.com/api/components/pcfData?from=2024-01-03T00%3A00%3A00.000Z&%20to=2024-12-31T23%3A59%3A59.999Z' \
     --header 'Content-Type: application/json' \
     --header 'Authorization: Bearer TOKEN' \

   ```

   You can retrieve the PCF data shared by a specific supplier.

   Example Request

   ```bash
     curl --location 'https://app.sigreen.siemens.com/api/components/pcfData?from=2024-01-03T00%3A00%3A00.000Z&%20to=2024-12-31T23%3A59%3A59.999Z&supplierId=0192416b-087d-7b2d-82b4-e08ad5da65db' \
     --header 'Content-Type: application/json' \
     --header 'Authorization: Bearer TOKEN' \

   ```

   You can retrieve the PCF data shared by the supplier wherein the PCF data is in any of the following state:

   **Accepted**

   Example Request

   ```bash
     curl --location 'https://app.sigreen.siemens.com/api/components/pcfData?supplierId=0192416b-087d-7b2d-82b4-e08ad5da65db&accepted=true' \
     --header 'Content-Type: application/json' \
     --header 'Authorization: Bearer TOKEN' \

   ```

   **Rejected**

   Example Request

   ```bash
     curl --location 'https://app.sigreen.siemens.com/api/components/pcfData?supplierId=019232dd-8e5d-72bf-9ed1-01fbf137cf6b&rejected=true' \
     --header 'Content-Type: application/json' \
     --header 'Authorization: Bearer TOKEN' \

   ```

   **Withdrawn**

   Example Request

   ```bash
     curl --location 'https://app.sigreen.siemens.com/api/components/pcfData?supplierId=019232dd-8e5d-72bf-9ed1-01fbf137cf6b&withdrawn=true' \
     --header 'Content-Type: application/json' \
     --header 'Authorization: Bearer TOKEN' \

   ```

   **Active**

   Example Request

   ```bash
     curl --location 'https://app.sigreen.siemens.com/api/components/pcfData?supplierId=0192416b-087d-7b2d-82b4-e08ad5da65db&active=true' \
     --header 'Content-Type: application/json' \
     --header 'Authorization: Bearer TOKEN' \

   ```

   **Expired**

   Example Request

   ```bash
     curl --location 'https://app.sigreen.siemens.com/api/components/pcfData?supplierId=01924911-87ea-7918-a753-9e3602c7d285&expired=true' \
     --header 'Content-Type: application/json' \
     --header 'Authorization: Bearer TOKEN' \

   ```

   **Revoked**

   Example Request

   ```bash
     curl --location 'https://app.sigreen.siemens.com/api/components/pcfData?supplierId=019232dd-8e5d-72bf-9ed1-01fbf137cf6b&revoked=true' \
     --header 'Content-Type: application/json' \
     --header 'Authorization: Bearer TOKEN' \

   ```

For more information in detail about the **Parameters**, **Request Body**, and **Responses** refer to the [API Reference](api-procurement.html) under Procurement API(s).

## Material Category

With the available Material Category APIs, you can perform the following operations:

1. Get Material Category details by ID.
1. Update Material Categories details by ID.
1. Delete Material Categories details by ID.
1. Get Material Category image by ID.
1. Update Material Category image by ID.
1. Delete Material Category image by ID.
1. Get all Material Categories for one company in SiGREEN.
1. Create Material Category to SiGREEN.

For more information in detail about the **Parameters**, **Request Body**, and **Responses** refer to the [API Reference](api-procurement.html) under Procurement API(s).

# SiGREEN Product API – API Overview

## Product API (SiGREEN Product version 1.16.9)

[Download OpenAPI Specification](api_specs/openapi-spec-v1.16.9.yaml)

## Product API (SiGREEN Product version 1.16.0)

[Download OpenAPI Specification](api_specs/openapi-spec-v1.16.0.yaml)

## Product API v2 (SiGREEN Product version 1.16.0)

[Download OpenAPI Specification](api_specs/openapi-spec-v2-1.16.0.yaml)

## Product API (Previous SiGREEN Product version 1.15.0)

[Download OpenAPI Specification](api_specs/openapi-spec-v1.15.0.yaml)

## Product API v2 (Previous SiGREEN Product version 1.15.0)

[Download OpenAPI Specification](api_specs/openapi-spec-v2-1.15.0.yaml)

# Changelog

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).

## [1.15.0] - 2025-06-23

#### What's New

______________________________________________________________________

##### `POST` /products/{id}/bomVersions

> Create a Bill of Material Versions for a product in SiGREEN

##### `DELETE` /products/{id}/factoryEmissions

> Deletes factory emissions data associated with a specific product in the SiGREEN system. The deletion requires a factory ID and can be further scoped using optional filters such as batch number, revision, and a date range to precisely manage and remove outdated or invalid emissions records.

##### `DELETE` /products/{id}/bomVersions/{bomVersionId}

> Delete a BOM version of a product in SiGREEN.

##### `PATCH` /products/{id}/bomVersions/{bomVersionId}

> Update the BOM version of a product in SiGREEN.

##### `GET` /products/{id}/bom/{bomVersionId}

> Get a list of Bill of Material entries for a product in SiGREEN.

##### `PATCH` /products/{id}/bom/{bomVersionId}

> Add Groups, Components, and Products to product BOM version in SiGREEN, and return the job Id. Job Status API can be used to retrieve the status of operations. This API performs tasks such as adding BOM items, synchronizing components, and executing calculations.

##### `DELETE` /products/{id}/bom/{bomVersionId}/items/{bomItemId}

> Delete BOM item (Groups,Components and Products)) to product BOM version in SiGREEN.

##### `PATCH` /products/{id}/bom/{bomVersionId}/items/{bomItemId}

> Update meta data of Groups,Components and Products to product BOM version in SiGREEN.

##### `PATCH` /products/{id}/bom/{bomVersionId}/saveUpstreamEmissions

> Save the current state of the upstream emission data for a product BOM version in SiGREEN. Once saved, the upstream emissions BOM will be locked, and no further changes will be allowed.

##### `GET` /products/jobs/{id}/status

> This API is used to fetch the status of a job

#### What's Deleted

______________________________________________________________________

##### `GET` /products/{productId}/bom

> Get a list Bill of Material entries for a product in SiGREEN

##### `POST` /products/{productId}/bom

> Create a Bill of Material for a product in SiGREEN

##### `POST` /products/{productId}/offsetting

> Offset your emissions.

#### Breaking Changes

______________________________________________________________________

##### `POST` /products/identifierTypes

###### Request:

Changed content type : `application/json`

New required properties:

- `mandatory`

- Added property `mandatory` (boolean)

  > Whether this identifier must be filled when creating products.

note

Please note that this change will only affect new data. Existing data will not be affected unless they are updated.

##### `PATCH` /products/identifierTypes/{id}

###### Request:

Changed content type : `application/json`

New required properties:

- `mandatory`

- Added property `mandatory` (boolean)

  > Whether this identifier must be filled when creating products.

note

Please note that this change will only affect new data. Existing data will not be affected unless they are updated.

##### `GET` /products/{id}/pcf

###### Return Type:

Changed response : **200 OK**

> Product carbon footprint of a product in a SiGREEN company

- Changed content type : `application/json`

  - Changed property `factoryEmission` (object)

    > Details about the factory emission.

    New optional properties:

    - `factoryId`

    - `from`

    - `productCarbonFootprint`

    - `to`

    - Added property `pcfIncludingBiogenic` (string)

      > Factory emission GWP total (incl. bio.). kg CO2e/declared unit

    - Added property `pcfExcludingBiogenic` (string)

      > Factory emission GWP total (excl. bio.). kg CO2e/declared unit

    - Deleted property `revision` (string)

      > Revision

    - Deleted property `factoryId` (string)

      > UUID to identify the factory of the company in SiGREEN

    - Deleted property `from` (string)

      > From date

    - Deleted property `to` (string)

      > to date

    - Deleted property `batch` (object)

      > Batch

    - Deleted property `productCarbonFootprint` (number)

      > Product carbon foot print

    - Deleted property `emissions` (array)

      > Factory emission activities

    - Deleted property `comment` (string)

      > Comments

    - Deleted property `sourceSystem` (string)

      > Source system

#### What's Changed

______________________________________________________________________

##### `GET` /products/identifierTypes

###### Return Type:

Changed response : **200 OK**

> List of product identifier types

- Changed content type : `application/json`

  - Changed property `items` (array)

    Changed items (object):

    New required properties:

    - `isMandatory`

    - Added property `isMandatory` (boolean)

      > Whether this identifier must be filled when creating products.

##### `POST` /products/identifierTypes

###### Request:

Changed content type : `application/json`

New required properties:

- `mandatory`

- Added property `mandatory` (boolean)

  > Whether this identifier must be filled when creating products.

##### `PATCH` /products/identifierTypes/{id}

###### Request:

Changed content type : `application/json`

New required properties:

- `mandatory`

- Added property `mandatory` (boolean)

  > Whether this identifier must be filled when creating products.

##### `GET` /products

###### Parameters:

Changed: `sort` in `query`

> Sorting fields supported (name, createdOn, updatedOn). Example - Ascending order '+name' , Descending order '-name'. Default order is createdOn ascending order.

###### Return Type:

Changed response : **200 OK**

> A list of products

- Changed content type : `application/json`

  - Changed property `items` (array)

    Changed items (object):

    - Changed property `identifiers` (array)
      > There should be one identifier where default parameter is set to true. That will act as main identifier for this entity.

##### `POST` /products

###### Request:

Changed content type : `application/json`

- Changed property `identifiers` (array)
  > There must be one identifier with the default parameter set to true, as it represents the main identifier for this entity.

##### `GET` /products/{id}

###### Return Type:

Changed response : **200 OK**

> A product

- Changed content type : `application/json`

  - Changed property `identifiers` (array)
    > There should be one identifier where default parameter is set to true. That will act as main identifier for this entity.

##### `PUT` /products/{id}

###### Request:

Changed content type : `application/json`

- Changed property `identifiers` (array)
  > There must be one identifier with the default parameter set to true, as it represents the main identifier for this entity.

##### `GET` /products/{id}/pcf

###### Return Type:

Changed response : **200 OK**

> Product carbon footprint of a product in a SiGREEN company

- Changed content type : `application/json`

  - Changed property `factoryEmission` (object)

    > Details about the factory emission.

    New optional properties:

    - `factoryId`

    - `from`

    - `productCarbonFootprint`

    - `to`

    - Added property `pcfIncludingBiogenic` (string)

      > Factory emission GWP total (incl. bio.). kg CO2e/declared unit

    - Added property `pcfExcludingBiogenic` (string)

      > Factory emission GWP total (excl. bio.). kg CO2e/declared unit

    - Deleted property `revision` (string)

      > Revision

    - Deleted property `factoryId` (string)

      > UUID to identify the factory of the company in SiGREEN

    - Deleted property `from` (string)

      > From date

    - Deleted property `to` (string)

      > to date

    - Deleted property `batch` (object)

      > Batch

    - Deleted property `productCarbonFootprint` (number)

      > Product carbon foot print

    - Deleted property `emissions` (array)

      > Factory emission activities

    - Deleted property `comment` (string)

      > Comments

    - Deleted property `sourceSystem` (string)

      > Source system

##### `GET` /products/bomVersions

###### Parameters:

Added: `locked` in `query`

> BOM is locked or not.true or false.

Added: `sort` in `query`

> Sorting fields supported (createdOn, updatedOn,upstreamEmissionCreatedOn). Example - Ascending order '+createdOn' , Descending order '-createdOn'. Default order is createdOn ascending order.

###### Return Type:

Changed response : **200 OK**

> Bill of Materials metadata

- Changed content type : `application/json`

  - Changed property `items` (array)

    Changed items (object):

    - Added property `id` (string)

      > UUID of product BOM Version of the company in SiGREEN

    - Added property `latestSync` (string)

      > Last BOM sync DateTime.DateTime (ISO 8601; UTC Timezone)

    - Added property `upstreamEmission` (object)

      > Details about the upstream emission details.

      - Property `pcfIncludingBiogenic` (string)
        > Upstream GWP total (incl. bio.). kg CO2e/declared unit
      - Property `pcfExcludingBiogenic` (string)
        > Upstream GWP total (excl. bio.). kg CO2e/declared unit

    - Added property `productionStage` (object)

      > Production stage details

      - Property `pcfIncludingBiogenic` (string)
        > Production stage GWP total (incl. bio.). kg CO2e/declared unit
      - Property `pcfExcludingBiogenic` (string)
        > Production stage GWP total (excl. bio.). kg CO2e/declared unit

    - Added property `distributionStage` (object)

      > Distribution stage details

      - Property `pcfIncludingBiogenic` (string)
        > Distribution stage GWP total (incl. bio.). kg CO2e/declared unit
      - Property `pcfExcludingBiogenic` (string)
        > Distribution stage GWP total (excl. bio.). kg CO2e/declared unit

    - Added property `upstreamEmissionCreatedOn` (string)

      > Emission createdOn.DateTime (ISO 8601; UTC Timezone)

    - Added property `createdOn` (string)

      > BOM version Created on (ISO 8601; UTC Timezone)

    - Added property `updatedOn` (string)

      > BOM version Updated on (ISO 8601; UTC Timezone)

##### `GET` /products/pcfData

> GET the PCF data for products in SiGREEN.

###### Parameters:

Changed: `sort` in `query`

> Sorting fields supported (createdOn). Example - Ascending order '+createdOn' , Descending order '-createdOn'. Default order is createdOn ascending order.

## [1.13.0] - 2025-02-10

### What's New

______________________________________________________________________

##### `GET` /products/pcfData

> GET the PCF data for your Products in SiGREEN.

##### `GET` /products/bomVersions

> Get a list of BOM versions

#### What's Deprecated

______________________________________________________________________

##### `POST` /products/{id}/factoryEmissions

> Upload factory emissions for a product in SiGREEN.

#### What's Changed

______________________________________________________________________

##### `POST` /products/identifierTypes

###### Return Type:

Changed response : **201 Created**

> Created. Response will return the UUID of the created product identifier type.

- Changed content type : `application/json`

  - Changed property `id` (string)
    > UUID to identify the identifier type of the company in SiGREEN

##### `DELETE` /products/identifierTypes/{id}

###### Parameters:

Changed: `id` in `path`

> UUID to identify the product identifier type of the company in SiGREEN.

##### `PATCH` /products/identifierTypes/{id}

###### Parameters:

Changed: `id` in `path`

> UUID for identifying the product identifier type of the company in SiGREEN.

##### `GET` /products

###### Parameters:

Added: `searchText` in `query`

> Partial text search on following attributes - name , identifiers.\
> Minimum 3 characters required to search.\
> Partial text match would be either name or identifiers or both

Added: `name` in `query`

> Filter products by name

Added: `family` in `query`

> Filter products by product family

Added: `idType` in `query`

> Product identifier type in SiGREEN.\
> idType filter is applicable only when user is searching with idValue filter.\
> Idea here is to find products with idType and value combination.

Added: `idValue` in `query`

> Filter products based on either main or additional identifier value in SiGREEN.

Added: `factoryId` in `query`

> UUID to identify the factory of the company in SiGREEN

Added: `sort` in `query`

> Sorting fields supported (name, createdOn, updatedOn). Example - Ascending order '+name' , Descending order '-name'. Default order is createdOn ascending order.

###### Return Type:

Changed response : **200 OK**

> A list of products

- Changed content type : `application/json`

  - Changed property `items` (array)

  Changed items (object):

  ```text
  * Added property `createdOn` (string)
    > Created on (ISO 8601; UTC Timezone)

  * Added property `updatedOn` (string)
      > Updated on (ISO 8601; UTC Timezone)

  * Changed property `id` (string)
      > UUID to identify the product of the company in SiGREEN

  * Changed property `factoryIds` (array)
      > Product Factory Ids (UUIDs)

  ```

##### `POST` /products

###### Request:

Changed content type : `application/json`

- Changed property `factoryIds` (array)
  > Product Factory Ids (UUIDs)

###### Return Type:

Changed response : **201 Created**

> Created

- Changed content type : `application/json`

  - Changed property `id` (string)
    > UUID to identify the product of the company in SiGREEN

##### `GET` /products/{id}

###### Parameters:

Changed: `id` in `path`

> UUID to identify the product of the company in SiGREEN

###### Return Type:

Changed response : **200 OK**

> A product

- Changed content type : `application/json`

  - Added property `createdOn` (string)

    > Created on (ISO 8601; UTC Timezone)

  - Added property `updatedOn` (string)

    > Updated on (ISO 8601; UTC Timezone)

  - Changed property `id` (string)

    > UUID to identify the product of the company in SiGREEN

  - Changed property `factoryIds` (array)

    > Product Factory Ids (UUIDs)

##### `PUT` /products/{id}

###### Parameters:

Changed: `id` in `path`

> UUID to identify the product of the company in SiGREEN

###### Request:

Changed content type : `application/json`

- Changed property `factoryIds` (array)
  > Product Factory Ids (UUIDs)

##### `DELETE` /products/{id}

###### Parameters:

Changed: `id` in `path`

> UUID to identify the product of the company in SiGREEN

##### `GET` /products/{id}/pcf

> Get product PCF details by id in SiGREEN.

###### Parameters:

Changed: `id` in `path`

> UUID to identify the product of the company in SiGREEN

###### Return Type:

Changed response : **200 OK**

> Product carbon footprint of a product in a SiGREEN company

- Changed content type : `application/json`

  - Added property `pcfId` (string)

    > The unique PCF Id of a credential

  - Added property `productionStage` (object)

    > Production stage details

    - Property `pcfIncludingBiogenic` (string)
      > Production stage GWP total (incl. bio.). kg CO2e/declared unit
    - Property `pcfExcludingBiogenic` (string)
      > Production stage GWP total (excl. bio.). kg CO2e/declared unit

  - Added property `distributionStage` (object)

    > Distribution stage details

    - Property `pcfIncludingBiogenic` (string)
      > Distribution stage GWP total (incl. bio.). kg CO2e/declared unit
    - Property `pcfExcludingBiogenic` (string)
      > Distribution stage GWP total (excl. bio.). kg CO2e/declared unit

  - Added property `upstream` (object)

    > Details about the upstream emission details.

    - Property `pcfIncludingBiogenic` (string)
      > Upstream GWP total (incl. bio.). kg CO2e/declared unit
    - Property `pcfExcludingBiogenic` (string)
      > Upstream GWP total (excl. bio.). kg CO2e/declared unit

  - Deleted property `totalPcf` (number)

    > Total Pcf

  - Deleted property `upstreamPcf` (number)

    > Upstream Pcf

  - Deleted property `dataQuality` (array)

    > Data Quality

  - Changed property `factoryEmission` (number -> object)

  - Changed property `active` (boolean)

    > The active status of the product

  - Changed property `creationDate` (string -> string)

    > The creation date of the product

  - Changed property `validityPeriodStart` (string -> string)

    > The start of the validity period of the product

  - Changed property `validityPeriodEnd` (string -> string)

    > The end of the validity period of the product

  - Changed property `certifier` (string)

    > The certifier of the product

  - Changed property `revision` (string)

    > The revision of the product

  - Changed property `bomVersion` (string)

    > The BOM version of the product

  - Changed property `emissionUnit` (string)

    > The emission unit of the product

    Removed enum values:

    - `piece`
    - `kg`
    - `m2`
    - `m3`
    - `kWh`
    - `tkm`
    - `MJ`
    - `L`
      - Changed property `quantity` (number -> number)
        > The quantity of the product

##### `POST` /factories

###### Return Type:

Changed response : **201 Created**

> Created. Response will return the UUID of the created factory.

- Changed content type : `application/json`

  - Changed property `id` (string)
    > UUID to identify the factory of a company in SiGREEN.

##### `DELETE` /factories/{id}

###### Parameters:

Changed: `id` in `path`

> UUID to identify the factory of the company in SiGREEN.

##### `PATCH` /factories/{id}

###### Parameters:

Changed: `id` in `path`

> UUID for identifying the factory of the company in SiGREEN.

##### `GET` /products/{productId}/bom

###### Parameters:

Changed: `productId` in `path`

> Alphanumeric Id (Product Identifier) for identifying the product of the company in SiGREEN

###### Return Type:

Changed response : **200 OK**

> Bill of Materials

- Changed content type : `application/json`

  - Changed property `boms` (array)

  Changed items (object):

  ```text
  * Changed property `version` (string)
    > Version of the Bill of Material

  * Changed property `comment` (string)
      > Comments

  * Changed property `revision` (string)
      > Revision of the product

  * Changed property `components` (array)
      > List of BOM Components of the product

      Changed items (object):

      * Changed property `identifier` (string)
          > Identifier of the component

      * Changed property `quantity` (string)
          > Quantity of the component

      * Changed property `components` (array)
          > List of BOM Components of the product

  ```

##### `POST` /products/{productId}/bom

###### Parameters:

Changed: `productId` in `path`

> Alphanumeric Id (Product Identifier) for identifying the product of the company in SiGREEN

###### Request:

Changed content type : `application/json`

- Changed property `version` (string)

  > Version of the Bill of Material

- Changed property `comment` (string)

  > Comments

- Changed property `revision` (string)

  > Revision of the product

- Changed property `components` (array)

  > List of BOM Components of the product

  Changed items (object):

  - Changed property `identifier` (string)
    > Identifier of the component
  - Changed property `quantity` (string)
    > Quantity of the component
  - Changed property `components` (array)
    > List of BOM Components of the product

##### `POST` /products/{productId}/offsetting

###### Parameters:

Changed: `productId` in `path`

> Alphanumeric Id (Product Identifier) for identifying the product of the company in SiGREEN

###### Request:

Changed content type : `application/json`

Changed items (object):

- Changed property `batchNumber` (string)

  > batch number

- Changed property `taskId` (string)

  > task id

- Changed property `offsetTotal` (number)

  > offset total

- Changed property `offsetUnit` (string)

  > offset unit

- Changed property `offsetPortfolioId` (string)

  > offset portfolio id

- Changed property `offsetPrice` (number)

  > offset price

- Changed property `offsetPriceUnit` (string)

  > offset price unit

- Changed property `offsetOwner` (string)

  > offset owner

- Changed property `offsetReason` (string)

  > offset reason

- Changed property `offsetOwnerVisibility` (string)

  > offset owner visibility

- Changed property `offsetReasonVisibility` (string)

  > offset reason visibility

- Changed property `offsetProviderOrderID` (string)

  > offset provider order id

- Changed property `certificateLink` (string)

  > certificate link

- Changed property `comment` (string)

  > Comments

- Changed property `offsetProjects` (array)

  > List of offset projects

  Changed items (object):

  - Changed property `offsetType` (string)
    > offset type
  - Changed property `offsetProvider` (string)
    > offset provider
  - Changed property `offsetProviderProjectID` (string)
    > offset provider project id
  - Changed property `offsetProviderFinalizationID` (string)
    > offset provider finalization id
  - Changed property `offsetProviderFinalizationDate` (string)
    > offset provider finalization date
  - Changed property `offsetQuantity` (number)
    > offset quantity
  - Changed property `offsetQuantityUnit` (string)
    > offset quantity unit
  - Changed property `projectID` (string)
    > project id
  - Changed property `projectName` (string)
    > project name
  - Changed property `projectLocation` (string)
    > project location
  - Changed property `projectType` (string)
    > project type
  - Changed property `projectURL` (string)
    > project url
  - Changed property `productType` (string)
    > product type
  - Changed property `offsetRetirementURL` (string)
    > offset retirement url
  - Changed property `offsetRetirementDate` (string)
    > offset retirement date
  - Changed property `offsetVintageStart` (string)
    > offset vintage start
  - Changed property `offsetVintageEnd` (string)
    > offset vintage end
  - Changed property `offsetSerialNumber` (string)
    > offset serial number
  - Changed property `offsetOwner` (string)
    > offset owner
  - Changed property `offsetReason` (string)
    > offset reason

##### `POST` /products/{id}/factoryEmissions

###### Parameters:

Changed: `id` in `path`

> UUID to identify the product of the company in SiGREEN

###### Request:

Changed content type : `application/json`

New required properties:

- `factoryId`

New optional properties:

- `emissions`

- Changed property `factoryId` (string)

  > UUID to identify the factory of the company in SiGREEN

- Changed property `batch` (object)

  > Batch

  New optional properties:

  - `assessmentYear`

###### Return Type:

Changed response : **201 Created**

> Created

- Changed content type : `application/json`

  - Changed property `factoryEmissionId` (string)
    > UUID to identify the factory emission of the company in SiGREEN

## [1.12.0] - 2024-10-14

### What's New

______________________________________________________________________

#### `DELETE` /products/{id}

> Delete a product in SiGREEN.

#### What's Changed

______________________________________________________________________

##### POST products/{id}/factoryEmissions

- API endpoint changed from "/products/factoryEmissions/{productId}" to "/products/{id}/factoryEmissions"
- Parameter changed from {productId} - product main identifier value to {id} - product system generated id
- Changed Request body structure changed from array to single object.
- Changed Request body - Instead of factory name now it uses factory id.

##### `PATCH` /factories/{id}

###### Parameters

Changed: `id` in `path`

> System generated id to identify the factory of the company in SiGREEN.

##### `GET` /products/{id}

Changed response : --200 OK--

> The request was successful and received single product data.

- Changed content type : `application/json`

  - Changed property `unitType` (string)
    > Unit type of the product

  Added enum values:

  ```text
  - `piece`
  - `kg`
  - `m2`
  - `m3`
  - `kWh`
  - `tkm`
  - `MJ`
  - `L`

  ```

##### `PUT` /products/{id}

###### Request

Changed content type : `application/json`

New required properties:

- `identifiers`

- Changed property `unitType` (string)

  > Unit type of the product

Added enum values:

```text
- `piece`
- `kg`
- `m2`
- `m3`
- `kWh`
- `tkm`
- `MJ`
- `L`

```

##### `GET` /products/identifierTypes

Changed response : --200 OK--

> List of product identifier types.

- Changed content type : `application/json`

  - Added property `totalRecords` (integer)

    > Total number of records.

  - Changed property `items` (array)

    > List of product identifiers

  Changed items (object):

  > Details about the Identifier Type

  ```text
  - Changed property `id` (string -> string)
    > System generated id to identify the identifier type present in a company in SiGREEN

  - Changed property `idType` (string)
    > Identifier type value.

  ```

##### `GET` /factories

###### Return Type

Changed response : --200 OK--

> List of factories belongs to a specific company.

- Changed content type : `application/json`

  - Added property `totalRecords` (integer)
    > Total number of records.

##### `POST` /products

###### Request

> Product details for create

Changed content type : `application/json`

New required properties:

- `identifiers`

- Changed property `unitType` (string)

  > Unit type of the product

Added enum values:

```text
- `piece`
- `kg`
- `m2`
- `m3`
- `kWh`
- `tkm`
- `MJ`
- `L`

```

##### `GET` /products

###### Return Type

Changed response : --200 OK--

> The request was successful and received list of product data.

- Changed content type : `application/json`

  - Added property `totalRecords` (integer)

    > Total number of records.

  - Changed property `items` (array)

    > List of products

  - Changed items (object):

    > Product response data.

    Changed property `unitType` (string)

    > Unit type of the product

    ```text
    Added enum values:

      - `piece`
      - `kg`
      - `m2`
      - `m3`
      - `kWh`
      - `tkm`
      - `MJ`
      - `L`

    ```

## [1.11.0] - 2024-07-22

### What's New

______________________________________________________________________

#### `POST` /products/identifierTypes

> Create a product identifier type for a company in SiGREEN.

##### `DELETE` /products/identifierTypes/{id}

> Delete an unused product identifier type for a company in SiGREEN.

##### `PATCH` /products/identifierTypes/{id}

> Update a product identifier type for a company in SiGREEN.

##### `DELETE` /factories/{id}

> Delete an unused factory for a company in SiGREEN.

##### `PATCH` /factories/{id}

> Update a factory for a company in SiGREEN.

##### `PUT` /products/{id}

> Update a product by product Id in SiGREEN

#### What's Changed

______________________________________________________________________

##### `GET` /products/{productId}/pcf/{filter}

> Get an export of a product carbon footprint without cryptographic material. If you need cryptographic verifiability of this information please use DIDComm APIs.

###### Parameters

Removed: `filter` in `path` Added: `filter` in `query`

> Here you can provide a filter on the request, e.g pcf information out of active or latest credential. Available values : active, latest

Changed path parameter : `productId` in `path` to `id` in `path`

##### `GET` /products/identifierTypes

> Get product identifier types for a company in SiGREEN.

###### Parameters

Added: `page` in `query`

> Page Number, usually starts with 0

Added: `size` in `query`

> Size of the page default value is 10

###### Return Type

New response : --403 Forbidden--

> Forbidden. Deleted response : --400 Bad Request-- Bad request. Changed response : --401 Unauthorized-- Unauthorized.

- New content type : `application/problem+json`

Changed response : --500 Internal Server Error--

> Unexpected error.

- New content type : `application/problem+json`

Changed response : --200 OK--

> List of product identifier types

- Changed content type : `application/json`

  - Changed property `hasNext` (boolean)

    > true if next page exists (page < totalPages)

  - Changed property `hasPrev` (boolean)

    > true if a previous page exists (page > 0)

  - Changed property `page` (number)

    > The current page-number (starts with 0)

  - Changed property `pageNumber` (number)

    > The current page-number (starts with 1)

  - Changed property `size` (number)

    > The size of elements in the response

  - Changed property `totalPages` (number)

    > The total available pages

  - Changed property `identifierTypes` (array)

  Changed items (object):

  New required properties:

  - `id`
  - `idType`
  - `isMainIdentifierType`

  ```text
  - Added property `id` (string)
    > A unique id for identifier type.

  - Changed property `idType` (string)
    > identifier type value.

  - Changed property `isMainIdentifierType` (boolean)
    > Whether it is main identifier type or not.

  ```

##### `GET` /products/{productId}

###### Return Type

New response : --403 Forbidden--

> Forbidden. Deleted response : --404 Not Found-- Not found. Changed response : --400 Bad Request-- Bad request.

- New content type : `application/problem+json`

Changed response : --401 Unauthorized--

> Unauthorized.

- New content type : `application/problem+json`

Changed response : --500 Internal Server Error--

> Unexpected error.

- New content type : `application/problem+json`

Changed response : --200 OK--

> A product

- Changed content type : `application/json`

New required properties -`id`

```text
- Added property `id` (string)
  > System generated id to identify the product of the company in SiGREEN

- Added property `factoryIds` (array)
  > Product Factory Ids (System Generated Ids)

  Items (string):

- Deleted property `location` (string)

- Deleted property `factory` (array)

- Changed property `name` (string)
  > Name of Product

- Changed property `family` (string)
  > Product Family name

- Changed property `description` (string)
  > Product description

- Changed property `weight` (string)
  > Product weight

- Changed property `quantity` (string)
  > Product Quantity

- Changed property `identifiers` (array)
  > You have to have one identifier where default parameter is set to true as this represents your main identifier for this entity.

  Changed items (object):

    - Changed property `idType` (string)
      > identifier type

    - Changed property `value` (string)
      > identifier value

    - Changed property `default` (boolean)
      > whether main identifier type or not

- Changed property `unitType` (string)
  > Unit Types

```

# Quick Start

For getting started with our Application and our APIs, kindly refer to [Getting Started](../getting-started.html).

For general information about the **Product Portfolio** functionality, refer to the respective section in the SiGREEN Help: [Product Portfolio](https://siemens-help.stonly.com/kb/en/product-portfolio-made-easy-183581).

## Concepts and Glossary

For detailed information about the concepts and terminology, please refer to [Concepts and Glossary](../concepts-glossary.html).

## Product API

The Product API, part of SiGREEN's suite of APIs, offers users a streamlined solution for managing products and BOM. With endpoints including POST Product and GET Product, users can seamlessly add and retrieve data related to products.

**Prerequisite**: In order to create a product, it is mandatory to create the product identifier type first.

Getting started with the **Product** APIs involves the following steps:

1. Create a product identifier type for a company in SiGREEN. The identifier type created initially in the beginning acts as the main identifier type. Example Request

   ````bash
   curl --location 'https://app.sigreen.siemens.com/api/products/identifierTypes' \
   --header 'Content-Type: application/json' \
   --header 'Authorization: Bearer TOKEN' \
   --data '{
       "idType": "Product ID"
   }'
   ```Upload factory emissions for a product in SiGREEN.

   Example Response

   ```bash
   {
       "id": "01908d72-9005-7241-8f5c-37dcf1e65b5f"
   }

   ````

1. Get product identifier types for a company in SiGREEN.

   Example Request

   ```bash
   curl --location 'https://app.sigreen.siemens.com/api/products/identifierTypes?size=100'
   --header 'Authorization: Bearer TOKEN' \

   ```

   Example Response

   ```bash
   {
       "items": [
           {
               "id": "01908d72-9005-7241-8f5c-37dcf1e65b5f",
               "idType": "Product ID",
               "isMainIdentifierType": true
           }
       ],
       "hasNext": true,
       "hasPrev": false,
       "page": 0,
       "pageNumber": 1,
       "size": 1,
       "totalPages": 1,
       "totalRecords": 1
   }

   ```

1. Create a factory for a company in SiGREEN (Optional).

   Example Request

   ```bash
   curl --location 'app.sigreen.siemens.com/api/factories' \
   --header 'Content-Type: application/json' \
   --header 'Authorization: Bearer TOKEN' \
   --data '{
       "factory": "Munich factory"
   }'

   ```

   Example Response

   ```bash
   {
     "id": "974ca1b0-7088-4f5f-97e6-1d44fa18f694"
   }

   ```

1. Get list of factories in a company in SiGREEN.

   Example Request

   ```bash
   curl --location 'https://app.sigreen.siemens.com/api/factories' \
   --header 'Authorization: Bearer TOKEN' \
   --data ''

   ```

   Example Response

   ```bash
   {
       "items": [
           {
               "id": "974ca1b0-7088-4f5f-97e6-1d44fa18f694",
               "factory": "Munich factory"
           }
       ],
       "hasNext": true,
       "hasPrev": false,
       "page": 0,
       "pageNumber": 1,
       "size": 1,
       "totalPages": 1,
       "totalRecords": 1
   }

   ```

1. Create a product.

   Example Request

   ```bash
   curl --location 'https://app.sigreen.siemens.com/api/products' \
   --header 'Content-Type: application/json' \
   --header 'Authorization: Bearer TOKEN' \
   --data '{
     "name": "Simatics PL7",
     "family": "Controls E1456",
     "identifiers": [
       {
         "value": "UPC-1",
         "idType": "Product ID",
         "default": true
       }
     ],
     "description": "Your product description",
     "weight": "1.533",
     "quantity": "5",
     "unitType": "kg",
     "factoryIds": [
       "974ca1b0-7088-4f5f-97e6-1d44fa18f694"
     ]
   }'

   ```

   Example Response

   ```bash
   {
     "id": "0190771f-2363-7de3-839f-df9131b93f37"
   }

   ```

1. Get product details by id in SiGREEN.

   Example Request

   ```bash
   curl --location --request GET 'https://app.sigreen.siemens.com/api/products/0190771f-2363-7de3-839f-df9131b93f37' \
   --header 'Content-Type: application/json' \
   --header 'Authorization: Bearer TOKEN' \
   --data ''

   ```

   Example Response

   ```bash
   {
       "id": "0190771f-2363-7de3-839f-df9131b93f37",
       "name": "Simatics PL7",
       "family": "Controls E1456",
       "identifiers": [
           {
               "value": "UPC-1",
               "idType": "Product ID",
               "default": true
           }
       ],
       "description": "Your product description",
       "weight": "1.533",
       "quantity": "5.0",
       "unitType": "kg",
       "factoryIds": [
           "974ca1b0-7088-4f5f-97e6-1d44fa18f694"
       ],
       "createdOn": "2022-11-20T08:30:00.000Z",
       "updatedOn": "2022-11-21T10:30:00.000Z"
   }      

   ```

With the other available Product APIs, you can perform the following operations:

1. Update a product identifier type for a company in SiGREEN.
1. Delete an unused product identifier type for a company in SiGREEN.
1. Update product details by id in SiGREEN.
1. Get all product details for a company in SiGREEN.
1. Delete a product in SiGREEN.
1. Upload factory emissions for a product in SiGREEN.
1. Get the PCF data for your products in SiGREEN.
1. Offset the emissions of a product in SiGREEN.

note

This endpoint is part of **Product API** and is **now deprecated**. It has been replaced with an improved version in **Product API v2**.

## Factory API(s)

With the other available Factory APIs, you can perform the following operations:

1. Update a factory for a company in SiGREEN.
1. Delete an unused factory of a company in SiGREEN.

## Bill of Material API(s)

With the available Bill of Material APIs, you can perform the following operations:

1. Get a list of Bill of Material entries for a product version in SiGREEN.
1. Add items (Groups, Components, Products) to a BOM version.
1. Save upstream emissions for a BOM version.
1. Update metadata of a BOM item.
1. Delete a BOM item from a BOM version.

The Bill of Materials (BOM) functionality has evolved from a simple read-only feature to a fully interactive and powerful BOM management system. This enhancement allows users to perform full Create, Read, Update, and Delete (CRUD) operations on both BOM versions and their associated items, bringing flexibility and control to how product structures are defined, maintained, and tracked.

At the core of this functionality is support for a wide range of **BOM entities**:

- **Groups**: Logical containers that can represent modules or assemblies. Groups support **multi-level nesting**, enabling the modeling of complex, hierarchical product structures.
- **Components**: Physical parts sourced from procurement systems. Users can define quantities for components but cannot alter core metadata sourced externally.
- **Products**: Internal products that can be linked to **Product Carbon Footprint (PCF)** data. For these, users can manage quantity, link PCF IDs, and toggle emission-related flags like `includeDistributionStage`.

Users can now **create and manage multiple BOM versions** for a single product. Each version can carry its own identifier (e.g., "v1.0", "prototype-3"), metadata (e.g., comments, timestamps), and emissions tracking data. This versioning system enables iterative product design, version comparison, and historical tracking — essential for sustainability compliance and audit readiness.

The API also supports **granular operations**:

- Add or remove individual BOM items (Groups, Components, Products)
- Update specific metadata fields (e.g., group name, component quantity, product PCF ID)
- Delete entire subtrees of items if needed

A crucial addition is the **upstream emissions finalization feature**. Once a BOM version is fully assembled, users can finalize it by saving upstream emissions. This operation locks the BOM, preventing further changes. Locking ensures data consistency for environmental reporting and downstream calculations, protecting against accidental edits.

To aid traceability, the API offers detailed **retrieval capabilities**, such as:

- Listing all BOM versions for a product
- Tracking timestamps like `createdOn`, `updatedOn`, and `latestSync`
- Accessing emissions metadata including upstream values and lifecycle footprints

These features make SiGREEN’s BOM API not just a tool for structuring product data but a robust platform for version control, emissions accountability, and supply chain integration. It empowers product teams, sustainability officers, and procurement engineers to collaborate more effectively with full visibility into both the structure and environmental impact of their products.

For more information in detail about the **Parameters**, **Request Body**, and **Responses** refer to the [API Reference](api-product.html) under Product API(s).
# SIMATIC AX documentation

# Contact

SIMATIC AX is currently in limited sales across selected countries within dedicated workflows.

Please contact your sales representative for further information or [get in touch with our experts](https://www.siemens.com/global/en/products/automation/industry-software/automation-software/simatic-ax.html#Contactus) – they’ll be happy to help.

- [Overview](overview.html)
- [Contact](contact.html)

# SIMATIC AX

Based on Visual Studio Code, SIMATIC AX offers state-of-the-art IT tools in a lean development environment for programming and maintaining SIMATIC PLCs.

Disclaimer

**SIMATIC AX** is in an **Early Access** phase, released for specifically selected productive use cases. **Siemens** reserves the right to select those use cases and internally assess where they are deployed. Speak to your local sales representative if you are interested in becoming an **Early Adopter** for **SIMATIC AX** or [learn more](https://siemens.com/simatic-ax) about it and get in touch with our experts.

## Documentation

The [SIMATIC AX documentation](https://docs.industrial-operations-x.siemens.cloud/p/simatic-ax) helps you get started and provides comprehensive guides, tutorials, and reference materials for working with SIMATIC AX.

## Community

If you have technical questions or want to stay up to date regarding the development of SIMATIC AX [join our community](https://community.mendix.com/link/space/simatic-ax).

## Trainings

There are lots of training options available. Depending on your learning goal, you can choose one or all of the following trainings:

- Get an overview over all the basics to get started developing your own PLC software. The videos are designed as a lecture and guide you through the environment of the toolchain and its basic features. It is a freemium offering of Sitrain: 
- The training on GitHub will help make you fit in working with the toolchain at your own speed. The learning goal is to acquire comprehensive knowledge and practical skills: 

## Open Source

The [SIMATIC AX GitHub Community](https://github.com/simatic-ax) is a platform where you can find additional contents for SIMATIC AX, for example tutorials, application examples, example libraries and code snippets.

Below is a list of selected repositories in the GitHub Community.

## Third-party frameworks

A prime example for a third party framework is the [**Industrial Automation Xpansion (IX) project**](https://github.com/ix-ax) from [**Modern Technology Systems**](https://www.mts.sk/en/) with their projects **AX#** and **AXOpen**.

## Useful Links

| Link                                                                                                                            | Description                                                                                |
| ------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------ |
| [Website](https://siemens.com/simatic-ax)                                                                                       | Learn more about SIMATIC AX and get in touch with our sales representatives to get access. |
| [Access](https://www.siemens.com/global/en/products/automation/industry-software/automation-software/simatic-ax.html#Contactus) | Get in touch with our experts to get access to SIMATIC AX.                                 |
| [Training](https://github.com/simatic-ax/learning-path)                                                                         | Learn how to work with SIMATIC AX.                                                         |
| [Exchange](https://community.mendix.com/link/space/simatic-ax)                                                                  | Exchange with others about SIMATIC AX.                                                     |
| [Develop](https://github.com/simatic-ax)                                                                                        | Develop and make use of existing solutions for SIMATIC AX.                                 |
# SIMATIC Connector for ROS - ROXSIE documentation

# Changelog

## 0.3.1 (2026-03-12)

### 💥 Breaking Changes

- Renaming from `ROSie` to `ROXSIE`

### ✨ Features

- Added support for `comments` and `constants` defined in ROS .msg files
- Added support for `SIMATIC Open Controller 3`
- Added RMW installation parameter to `build_deploy_container.sh` for generated ROS 2 workspace
- Added command line message to refer to generated log file in case of a generation error

### 🔌 API Changes

- Renaming of headers from `rosie_api` to `roxsie_api`
- Renaming of namespace from `ROSie` to `ROXSIE`
- Renaming of ROS 2 node from `ROSieNode` to `ROXSIENode`
- Renaming of client variable from `_rosie_client` to `_roxsie_client`

### 🐛 Bug Fixes

- Adapted installation script to handle empty user inputs
- Carriage return characters will be filtered out from ROS 2 .msg definitions
- Added IEC check for all generated SCL code blocks in TIA Portal
- Fixed generation failure for copying custom interface packages to generated ROS 2 workspace

## 0.3.0 (2025-09-05)

### ✨ Features

- Added support for `RIB v2.2.4`
- Increased usability by `automatically creating a complete ROS 2 workspace` including the generated ROS 2 package (with all custom interface packages if defined) and docker deploy files
- Added `video tutorial` and `engineering system guide` sections to documentation for more user friendly setup and usage descriptions
- Modified docker build scripts to give the user more control for specifying the ROS 2 version and container name for their generated ROS 2 package
- Renamed generated `input` directory to `yaml_input` within output folder
- Added automatic search for ROS 2 Jazzy and other custom ROS 2 installations within the ROSie installer script
- Improved code generation error handling by providing additional information via `.log` file in the generated `environment` directory

### 🔌 API Changes

- Changed ROSie API header file `rosie.hpp` to provide a non-copyable ROSie client class, as objects should not be copied because of possible shared memory issues
- Adapted ROSie API header file `rosie_return_value.hpp` return type to be of type `struct ReturnValue` instead of `std::pair` for more readable return value handling

## 0.2.0 (2025-05-08)

### ✨ Features

- Added support for `ROS 2 Jazzy`
- Some changes in the generated .scl code file (e.g. valid elements counter prefix to `i_valid_elements_*`) to not violate the TIA style guide
- Added GIF's to documentation for more user friendly setup and usage descriptions

## 0.1.1 (2024-11-15)

### ✨ Features

- Removed deprecated optional PLC arguments (`cyclic_interrupt_ob_name`, `read_db_name`, `write_db_name`) from YAML config
- Switched to HTML-based interactive user documentation (not yet released)
- Improved runtime performance on PLC side for topics not containing strings
- Improved installation process
- Updated generator base docker image
- Improved error messages when user provided a non-compliant YAML config file
- Added checks for maximum allowed topic rate
- Updated log messages
- Added deployment application example for generated ROS 2 packages based on docker
- Added checks for topic names
- Added checks for maximum memory usage
- Added checks for limit of OBs and RIB DBs
- Fixed issue with failed generation process (missing header `rosidl_typesupport_cpp/macros.h`) if user provides standard ROS 2 installation instead of a source build
- Various bugfixes and performance improvements of the generator

## 0.1.0 (2024-08-30)

### ✨ Features

- First functional Minimum Viable Product (MVP) release based on `RIB v2.1.1` and `ROS 2 Humble`
- Includes support for ROS 2 publishers & subscribers with all standard and custom message types

# Contact

To get `ROXSIE` through your regional Siemens region, please visit our [Siemens Industry Online Support (SIOS) page](https://siemens.com/ros).

# Limitations

## String Size

The RIB only supports fixed size arrays of bytes to store string like texts.\
Therefore, ROS 2 message types that contain strings e.g. `string frame_id` in [std_msgs/msg/Header](https://github.com/ros2/common_interfaces/blob/jazzy/std_msgs/msg/Header.msg) will translate those strings to a `fixed size` byte array with the size of `254 bytes` (as shown in the picture below).\
⚠️ As the byte array will be available as a string on the PLC side, it is also converted on the PLC side. The PLC based string data type defines the maximum string length of 254 bytes.

## Vector Size

As the RIB does not provide dynamic SHM allocation during runtime, all ROS 2 based `vector` types are currently limited to a `fixed size of 64 elements`.\
⚠️ Additionally, a `valid elements counter` variable will be added to the PLC side, so that the user can define, how many valid elements are stored in the vector, if the number is smaller than 64.

## Shared Memory

As the RIB uses shared memory to access data through Industrial OS as well as the Software Controller (SW-PLC),\
The actual size of the used shared memory area is therefore limited and has to be conform to the memory limit of the SW-PLC.

An example usage is shown in the picture above, where the two ROS 2 message types [geometry_msgs/msg/Twist](https://github.com/ros2/common_interfaces/blob/jazzy/geometry_msgs/msg/Twist.msg) and [sensor_msgs/msg/BatteryState](https://github.com/ros2/common_interfaces/blob/jazzy/sensor_msgs/msg/BatteryState.msg) are used to provide data to the PLC (`ros2_to_plc`) via shared memory.

Those two definitions are combined to one shared memory area during runtime, that shall not exceed the actual SW-PLC memory limitations:

| SW-PLC Type      | v30.1 Code memory limit [MB] | v31.1 Code memory limit [MB] | v40.0 Code memory limit [MB] |
| ---------------- | ---------------------------- | ---------------------------- | ---------------------------- |
| S7-1505 SP       | 2                            | 2                            | 3                            |
| S7-1505 SP F     | 2                            | 2                            | 4.5                          |
| S7-1505 SP T (F) | 3                            | 3                            | 4.5                          |
| S7-1507 S (F)    | 5                            | 5                            | —                            |
| S7-1508 S (F)    | 10                           | 10                           | —                            |
| S7-1508 T (F)    | 12.5                         | 12.5                         | —                            |

⚠️ Note: As those limitations are defining the total PLC code memory limit, the used ROS 2 .msg typse that needs to be transfered to the PLC or vice versa should not be too big regarding memory layout.

## Number of RIB Clients

The RIB and its participant manager `RIB_App` supports up to 128 client applications.\
But as the generated ROS 2 package uses a precompiled RIB API library, currently only `one ROS 2 client connection` is possible, `combined with one SW-PLC based connection`.

## Number of Data Blocks / Symbols

The SW-PLC supports in total `up to 20 Cyclic Interrupt Organization Blocks (OBs)`.\
As each topic in the YAML configuration defines one Cyclic Interrupt OB (ROS 2 -> PLC and vice versa), the user needs to make sure that the provided YAML defines less than 20 topics in total or up to 16 topics for each side (ROS 2 -> PLC and vice versa) as shown in the picture below.

⚠️ The user has to keep in mind that the defined topics and therefore Cyclic Interrupt OBs will be added to already existing ones.

Additionally, the RIB provides the following maximum specifications:

| RIB      | Max. number of symbols (topics) | Max. number of provide / consume DBs |
| -------- | ------------------------------- | ------------------------------------ |
| `v2.1.1` | 1024                            | 16                                   |
| `v2.2.4` | 8192                            | 1024                                 |

## Comments

ROS 2 `.msg` files may contain comments using the `#` character.\
ROXSIE carries these comments over into the generated SCL code, if they follow one of the two supported syntaxes:

**Inline comment**: A `#` directly after a variable definition on the same line, followed by the comment text:

```bash
float32 voltage          # Voltage in Volts (Mandatory)

```

**Continuation comment**: A line consisting only of leading whitespace followed by `#`, continuing the comment of the variable + comment defined on the previous line:

```bash
float32[] cell_voltage   # An array of individual cell voltages for each cell in the pack
                         # If individual voltages unknown but number of cells known set each to NaN

```

Both styles are demonstrated in the ROS 2 standard interface package: [sensor_msgs/msg/BatteryState.msg](https://github.com/ros2/common_interfaces/blob/jazzy/sensor_msgs/msg/BatteryState.msg).

⚠️ Comments that do not follow either of the above formats — for example, standalone comment blocks not attached to a variable, or continuation lines missing the required leading whitespace — will not be included in the generated SCL code.

## Constants

ROS 2 `.msg` files support the definition of typed constants by assigning a fixed value to a named identifier using the `=` sign.\
ROXSIE recognizes these constant definitions and generates corresponding constant declarations in the generated SCL code.

A ROS constant follows the syntax `  = `, optionally followed by an inline comment:

```bash
uint8 POWER_SUPPLY_TECHNOLOGY_UNKNOWN = 0 # Unknown battery technology
uint8 POWER_SUPPLY_TECHNOLOGY_NIMH = 1    # Nickel-Metal Hydride battery

```

By convention, constant names are written in `UPPER_SNAKE_CASE` to distinguish them from regular variable definitions.\
Inline comments on constant lines are carried over into the generated SCL code using the same rules as described in the [Comments](#comments) section above.

Full examples of constant definitions can be found in the ROS 2 standard interface package: [sensor_msgs/msg/BatteryState.msg](https://github.com/ros2/common_interfaces/blob/jazzy/sensor_msgs/msg/BatteryState.msg).

⚠️ Constants are **read-only** values and will not be mapped to any RIB shared memory — they are purely informational declarations available on the PLC side.

## Topic Rates

At this stage, the RIB connections provided in the YAML config should `not exceed` a specified rate of `1000 Hz`.\
⚠️ The jitter for a defined rate highly depends on the complexity of the provided data type. For example, a [geometry_msgs/msg/Twist](https://github.com/ros2/common_interfaces/blob/jazzy/geometry_msgs/msg/Twist.msg) is able to reach a stable rate of 1 kHz, but a more complex data type such as [trajectory_msgs/msg/MultiDOFJointTrajectory](https://github.com/ros2/common_interfaces/blob/jazzy/trajectory_msgs/msg/MultiDOFJointTrajectory.msg) with its vector of strings and nested structs will cause a loss of data at this rate or exceeding the PLC cycle time causing the PLC to stop!\
⚠️ The usage of `strings` and/or `vectors` and/or `nested structs` of them `should be avoided at higher frequencies (>100 Hz)` as their conversion is time consuming!

## Generated ROS 2 Package

The generated C++ based ROS 2 package has the following properties:

- The generated ROXSIE API C++ code is `not thread safe` (as the RIB connection does not provide thread safe shared memory access).
- The global RIB cycle time for ROS 2 values written to the RIB will be set to the highest of all topic frequencies defined in the provided YAML config file.
- The ROXSIE API code is based on the `C++17` standard.
- The ROXSIE generated ROS 2 code is only tested with `Humble` and `Jazzy`.
- The ROS 2 data type `wstring` is currently `not supported`, but all other primitive ROS 2 types.

## Type Conversions

The following table shows the available standard data types and their corresponding data types in ROS 2, RIB and TIA.

| ROS 2 data type | RIB data type | TIA data type | Comment                                                         |
| --------------- | ------------- | ------------- | --------------------------------------------------------------- |
| `bool`          | RIB_BOOL      | Bool          |                                                                 |
| `byte`          | RIB_BYTE      | Byte          |                                                                 |
| `char`          | RIB_BYTE      | Char          |                                                                 |
| `float32`       | RIB_FLOAT     | Real          |                                                                 |
| `float64`       | RIB_DOUBLE    | LReal         |                                                                 |
| `int8`          | RIB_INT8      | SInt          |                                                                 |
| `int16`         | RIB_INT16     | Int           |                                                                 |
| `int32`         | RIB_INT32     | DInt          |                                                                 |
| `int64`         | RIB_INT64     | LInt          |                                                                 |
| `uint8`         | RIB_UINT8     | USInt         |                                                                 |
| `uint16`        | RIB_UINT16    | UInt          |                                                                 |
| `uint32`        | RIB_UINT32    | UDInt         |                                                                 |
| `uint64`        | RIB_UINT64    | ULInt         |                                                                 |
| `string`        | RIB_BYTE[254] | String        | The maximum string length is currently capped at 254 characters |
| `wstring`       | unsupported   | unsupported   | Wide strings are not supported yet                              |

For arrays the conversion is:

| ROS 2 data type | RIB data type | TIA data type   | Comment |
| --------------- | ------------- | --------------- | ------- |
| [10]            | [10]          | Array of [0..9] |         |

For vectors the conversion is:

| ROS 2 data type | RIB data type | TIA data type       | Comment                                                                                                                                                                           |
| --------------- | ------------- | ------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| []              | [64]          | Array of [0..63] of | Vectors will be converted to arrays of a fixed size (currently storing 64 elements)                                                                                               |
|                 | RIB_UINT32    | UDInt               | Additionally a `i_valid_elements_*` variable will be created for each ROS 2 vector on the RIB and TIA side to track the actual elements the ROS 2 vector contains / shall contain |

⚠️ Transferring ROS 2 vectors with more elements than the maximum size (current default is 64) will lead to data loss.

- [Overview](overview.html)
- [What is ROS & PLCs](ros_plc_overview.html)
- [Requirements](requirements.html)
- [Setup](setup/setup.html)
  - [Engineering System Guide](setup/engineering_system.html)
- [Usage](usage/usage.html)
  - [Command Line Tool](usage/additional_resources/command_line_tool.html)
  - [YAML Configuration Structure](usage/additional_resources/yaml_config_structure.html)
- [Video Tutorial](video_tutorial.html)
- [Limitations](limitations.html)
- [Troubleshooting](troubleshooting.html)
- [Contact](contact.html)
- [Changelog](CHANGELOG.html)
- [Archive]
  - [v0.3.0]
    - [Overview](archive/v0.3.0/overview.html)
    - [Requirements](archive/v0.3.0/requirements.html)
    - [What is ROS & PLCs](archive/v0.3.0/ros_plc_overview.html)
    - [Setup](archive/v0.3.0/setup/setup.html)
      - [Engineering System](archive/v0.3.0/setup/engineering_system.html)
    - [Usage](archive/v0.3.0/usage/usage.html)
      - [Additional Resources]
        - [Command Line Tool](archive/v0.3.0/usage/additional_resources/command_line_tool.html)
        - [YAML Configuration Structure](archive/v0.3.0/usage/additional_resources/yaml_config_structure.html)
    - [Video Tutorial](archive/v0.3.0/video_tutorial.html)
    - [Limitations](archive/v0.3.0/limitations.html)
    - [Troubleshooting](archive/v0.3.0/troubleshooting.html)
  - [v0.2.0]
    - [Overview](archive/v0.2.0/overview.html)
    - [Setup](archive/v0.2.0/setup.html)
    - [Usage](archive/v0.2.0/usage/usage.html)
      - [Command Line Tool](archive/v0.2.0/usage/command_line_tool.html)
      - [YAML Configuration Structure](archive/v0.2.0/usage/yaml_config_structure.html)
    - [Limitations](archive/v0.2.0/limitations.html)
    - [Troubleshooting](archive/v0.2.0/troubleshooting.html)

# SIMATIC Connector for ROS - ROXSIE

⚡**Disclaimer:** We renamed `ROSie` to `ROXSIE`!

[ROXSIE](https://siemens.com/ros) is a command-line-based code generation tool designed to facilitate the integration of ROS 2 with SIMATIC Software PLC connections based on shared memory. It operates based on a YAML configuration file to generate the required PLC and ROS 2 code.

✔️ It allows you to have a connection between your ROS 2 runtime and your PLC system, which are both coexisting, running and sharing information among each other inside the same Siemens industrial hardware, combining the [advantages of both worlds](ros_plc_overview.html#integrating-plcs-with-ros)!

The process to integrate the communication between the systems consist of two steps:

**1. Generate SCL Code:** Generator creates a SCL source file that needs to be imported into your TIA-Portal project. It contains all program blocks required for the data exchange with ROS 2.\
**2. Generate ROS 2 package:** A fully functional ROS 2 package is generated, which contains the connection to the Software PLC.

After the generation process, both programs can be executed on [suitable devices](requirements.html#2-runtime-system-requirements).

# Requirements

To utilize ROXSIE there are two systems involved:

1. **Engineering System**: Used to run the ROXSIE generator (e.g., a Windows engineering machine with Linux as subsystem).
1. **Runtime System**: Used to execute the generated code (e.g., a SIMATIC Industrial OS system with RIB and Software PLC).

## 1. Engineering System Requirements

### Hardware

- Any x86/x64 based system

### Software (all of the listed)

- [ROXSIE](https://siemens.com/ros) container running in a Linux environment with access to ROS 2 installation via [ENV variables](usage/usage.html#2-start-roxsie-container)
- Linux (e.g. [SIMATIC Industrial OS](https://sieportal.siemens.com/en-ww/products-services/10355854), Ubuntu, Debian) + Windows 10/11 (for TIA Portal) operating systems
- [ROS 2](https://github.com/ros2) running natively in Linux or in a container if installation is mounted to host to be accessible by ROXSIE container
- [Docker](https://www.docker.com/) (+ Docker Compose plugin) `>= v20.x.y`
- [TIA Portal](https://www.siemens.com/global/en/products/automation/industry-software/automation-software/tia-portal.html) running in Windows 10/11

⚠️ **Note:** To be able to use the RIB with Software Controller `v31.1`, TIA Portal v20 `>= Update 1` needs to be installed.

The use of virtual machines or [Windows Subsystem for Linux (WSL)](https://learn.microsoft.com/en-us/windows/wsl/install) is recommended in order to have TIA Portal and ROS 2 running in parallel in the same system.\
Please refer to the [Engineering System Setup](setup/engineering_system.html) for a detailed step by step instruction on how to prepare your engineering system.

## 2. Runtime System Requirements

### Hardware (one of the listed)

| Device | Name                                                                                                                                                                                                                        | Shipped IndOS Version | Shipped SW-PLC Version           | Recommendations                                                                                             |
| ------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------- | -------------------------------- | ----------------------------------------------------------------------------------------------------------- |
|        | [SIMATIC ET 200SP Open Controller CPU 1515SP PC2](https://sieportal.siemens.com/en-ww/products-services/detail/6ES7677-2DB43-0GB1) - [(F)](https://sieportal.siemens.com/en-ww/products-services/detail/6ES7677-2SB43-0GB1) | v3.x.y                | v3x.y                            | 🔋 **Embedded / Low Power** - Ideal for compact installations - Direct IO modules access                    |
|        | [SIMATIC ET 200SP Open Controller CPU 1515SP PC3](https://sieportal.siemens.com/en-ww/products-services/detail/6ES7677-2DD43-0GB0) - [(F)](https://sieportal.siemens.com/en-ww/products-services/detail/6ES7677-2SD43-0GB0) | v4.x.y                | v4x.y                            | 🔋 **Embedded / Low Power** - Successor to PC2 - Ideal for compact installations - Direct IO modules access |
|        | [SIMATIC IPC 427E](https://sieportal.siemens.com/en-ww/products-services/detail/6AG4141-.....-....)                                                                                                                         | v3.x.y                | — (must be purchased separately) | ⚖️ **Balanced Performance** - Great compromise between performance and efficiency                           |
|        | [SIMATIC IPC BX-39A](https://sieportal.siemens.com/en-ww/products-services/detail/6AG4142-.....-....)                                                                                                                       | v4.x.y                | — (must be purchased separately) | ⚖️ **Balanced Performance** - Successor to 427E - Great compromise between performance and efficiency       |

All of the listed devices have been tested and verified to work with ROXSIE generated code.\
This list is non-exhaustive and other devices might work with ROXSIE too as long as they fulfil the following software requirements.

### Software (all of the listed)

- [SIMATIC Industrial OS](https://sieportal.siemens.com/en-ww/products-services/10355854)
- [Realtime Information Backbone (RIB)]()
- [SIMATIC S7-1500 Software Controller (PLC)](https://sieportal.siemens.com/en-ww/products-services/10268311)
  - CPU 1505SP (F) pre-installed on the Open Controller
  - [CPU 1507S](https://sieportal.siemens.com/en-ww/products-services/10563102) - [(F)](https://sieportal.siemens.com/en-ww/products-services/10563191) and [CPU 1508S](https://sieportal.siemens.com/en-ww/products-services/10562981) - [(F)](https://sieportal.siemens.com/en-ww/products-services/10563192) are independent products that need to be bought in addition to an IPC
- [ROS 2](https://github.com/ros2) (supported: [Humble](https://docs.ros.org/en/humble/index.html), [Jazzy](https://docs.ros.org/en/jazzy/index.html)) installation to run the generated ROS 2 package (native or via container)

## Software Version Compatibilities

The following table shows the supported software versions of all ROXSIE releases:

| ROXSIE   | Software Controller                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           | TIA Portal                                                                                                                                                                                                                                                                                                                                                                | Industrial OS                                                                                                                                                                                                                                                                                         | RIB            | ROS 2 (tested)                                                                                         |
| -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------- | ------------------------------------------------------------------------------------------------------ |
| `v0.1.0` | [v30.1]()                                                                                                                                                                                                                                                                                                                                   | [v19](https://support.industry.siemens.com/cs/document/109820994/simatic-step-7-inkl-safety-s7-plcsim-and-wincc-v19-trial-download)                                                                                                                                                                                                                                       | [v3.4.2](https://support.industry.siemens.com/cs/document/109825741/simatic-industrial-os-v3-4-2)                                                                                                                                                                                                     | v2.1.1         | [Humble](https://docs.ros.org/en/humble/index.html)                                                    |
| `v0.1.1` | [v30.1]()                                                                                                                                                                                                                                                                                                                                   | [v19](https://support.industry.siemens.com/cs/document/109820994/simatic-step-7-inkl-safety-s7-plcsim-and-wincc-v19-trial-download)                                                                                                                                                                                                                                       | [v3.4.2](https://support.industry.siemens.com/cs/document/109825741/simatic-industrial-os-v3-4-2)                                                                                                                                                                                                     | v2.1.1         | [Humble](https://docs.ros.org/en/humble/index.html)                                                    |
| `v0.2.0` | [v30.1]()                                                                                                                                                                                                                                                                                                                                   | [v19](https://support.industry.siemens.com/cs/document/109820994/simatic-step-7-inkl-safety-s7-plcsim-and-wincc-v19-trial-download)                                                                                                                                                                                                                                       | [v3.4.2](https://support.industry.siemens.com/cs/document/109825741/simatic-industrial-os-v3-4-2)                                                                                                                                                                                                     | v2.1.1         | [Humble](https://docs.ros.org/en/humble/index.html), [Jazzy](https://docs.ros.org/en/jazzy/index.html) |
| `v0.3.0` | [v30.1](), [v31.1]()                                                                                                                                                             | [v19](https://support.industry.siemens.com/cs/document/109820994/simatic-step-7-inkl-safety-s7-plcsim-and-wincc-v19-trial-download), [v20](https://support.industry.siemens.com/cs/document/109963850/simatic-step-7-inkl-safety-s7-plcsim-und-wincc-v20-trial-download) >= [Update 1](https://support.industry.siemens.com/cs/document/109963851/tia-portal-v20-updates) | [v3.4.2](https://support.industry.siemens.com/cs/document/109825741/simatic-industrial-os-v3-4-2), [v3.5.3](https://support.industry.siemens.com/cs/document/109983416/simatic-industrial-os-v3-5-3)                                                                                                  | v2.1.1, v2.2.4 | [Humble](https://docs.ros.org/en/humble/index.html), [Jazzy](https://docs.ros.org/en/jazzy/index.html) |
| `v0.3.1` | [v30.1](), [v31.1](), [v40.0]() | [v19](https://support.industry.siemens.com/cs/document/109820994/simatic-step-7-inkl-safety-s7-plcsim-and-wincc-v19-trial-download), [v20](https://support.industry.siemens.com/cs/document/109963850/simatic-step-7-inkl-safety-s7-plcsim-und-wincc-v20-trial-download) >= [Update 1](https://support.industry.siemens.com/cs/document/109963851/tia-portal-v20-updates) | [v3.4.2](https://support.industry.siemens.com/cs/document/109825741/simatic-industrial-os-v3-4-2), [v3.5.3](https://support.industry.siemens.com/cs/document/109983416/simatic-industrial-os-v3-5-3), [v4.2.0](https://support.industry.siemens.com/cs/document/109989147/simatic-industrial-os-v4-2) | v2.1.1, v2.2.4 | [Humble](https://docs.ros.org/en/humble/index.html), [Jazzy](https://docs.ros.org/en/jazzy/index.html) |

Additionally, each Software Controller version requires a specific TIA Portal, Industrial OS and RIB version:

| Software Controller                                                                                                                                                           | TIA Portal                                                                                                                                                                                                                           | Industrial OS                                                                                     | RIB    |
| ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------- | ------ |
| [`v30.1`]() | [v19](https://support.industry.siemens.com/cs/document/109820994/simatic-step-7-inkl-safety-s7-plcsim-and-wincc-v19-trial-download)                                                                                                  | [v3.4.2](https://support.industry.siemens.com/cs/document/109825741/simatic-industrial-os-v3-4-2) | v2.1.1 |
| [`v31.1`]()        | [v20](https://support.industry.siemens.com/cs/document/109963850/simatic-step-7-inkl-safety-s7-plcsim-und-wincc-v20-trial-download) >= [Update 1](https://support.industry.siemens.com/cs/document/109963851/tia-portal-v20-updates) | [v3.5.3](https://support.industry.siemens.com/cs/document/109983416/simatic-industrial-os-v3-5-3) | v2.2.4 |
| [`v40.0`]()                  | [v20](https://support.industry.siemens.com/cs/document/109963850/simatic-step-7-inkl-safety-s7-plcsim-und-wincc-v20-trial-download) >= [Update 4](https://support.industry.siemens.com/cs/document/109963851/tia-portal-v20-updates) | [v4.2.0](https://support.industry.siemens.com/cs/document/109989147/simatic-industrial-os-v4-2)   | v2.2.4 |

# Overview of ROS and PLC Technology

## Programmable Logic Controller (PLC)

[Programmable Logic Controllers (PLCs)](https://www.siemens.com/global/en/products/automation/systems/industrial/plc.html) are robust industrial computers specifically designed for automating manufacturing processes. They are widely deployed in factory automation, industrial control systems, and production lines where reliability and consistent performance are critical.\
PLCs incorporate robust safety features including failsafe operation, safety-certified hardware variants, and compliance with industrial safety standards (IEC 61508/61131).

- **Key Features:**
  - Real-time, deterministic operation
  - High reliability and fault tolerance
  - Designed for 24/7 operation in harsh environments
  - Robust safety features
- **Programming:** Ladder Logic, Structured Text, Function Block Diagram
- **Strengths:** Reliability, stability, industrial standards compliance, low-code programming
- **Limitations:** Limited computational power, specialized programming knowledge, specialized IDE

## Robot Operating System (ROS)

The [Robot Operating System (ROS)](https://www.ros.org) is a flexible framework and middleware that facilitates robot software development. It has become the standard platform for research robots, autonomous vehicles, and service robots where adaptability and access to advanced capabilities are required.

- **Key Features:**
  - Distributed computing architecture
  - Rich ecosystem of tools and packages
  - Hardware abstraction capabilities
- **Programming:** C++, Python & others with node-based communication model
- **Strengths:** Flexibility, extensive libraries, active open-source community
- **Limitations:** No safety, not inherently real-time, complexity, steeper learning curve

## Integrating PLCs with ROS

Integrating PLCs with ROS creates a powerful synergy that leverages the strengths of both technologies.\
PLCs contribute industrial-grade reliability, deterministic operation, and safety certification essential for critical manufacturing processes, while ROS provides advanced capabilities in motion planning, perception, and AI and gives access to latest research results.

This combination enables sophisticated robotic applications that maintain industrial robustness - allowing manufacturers to implement complex automation solutions like adaptive manufacturing, intelligent material handling, and flexible production systems without sacrificing the reliability demanded by industrial environments.

# Troubleshooting

## Environment Variable Errors

This kind of error occurs, if the user did not setup the host environment properly e.g. via the [setup guideline](setup/setup.html) or manually by setting the host environment variables in the `.env file` accordingly, e.g. like shown below.

```bash
# [REQUIRED]
# The users ROS 2 base installation path share directory that contains all standard interfaces including their .msg files.
# This directory has to be be mounted in the ROXSIE container to be able to lookup all standard types.
# Standard ROS 2 installation example: /opt/ros/jazzy
# Custom ROS 2 installation example:   /home/${USER}/jazzy_ws
USER_ROS2_INSTALLATION_PATH=/opt/ros/jazzy

# [OPTIONAL]
# The user directory where the optionally needed custom ROS 2 interfaces are installed.
# Example: /home/user/ros2_ws
USER_ROS2_INTERFACE_INSTALLATION_PATH=/home/user/ros2_ws

# [REQUIRED]
# The current directory where this .env file and the whole provided ROXSIE workspace is located.
# This directory should contain an  directory that stores the config.yaml files and an  directory where the generated code is stored.
# Example: /home/${USER}/roxsie_workspace
USER_ROXSIE_WORKSPACE_PATH=/home/user/roxsie_workspace

```

### USER_ROXSIE_WORKSPACE_PATH

The `USER_ROXSIE_WORKSPACE_PATH` environment variable defines the path to your actual ROXSIE workspace. Here, the `input` and `output` directories as well as the `docker-compose.yaml` and `.env` files should be located.

#### ⚠️ Variable Not Defined

If you did not define the ROXSIE workspace, where the input/output directories are located, the following error will be displayed:

```bash
$ roxsie -c example_config.yaml
####################################  ROXSIE  ###################################
#  Version: 0.3.1                                                               #
#  Date:    dd.mm.YYYY                                                          #
#  Time:    HH:MM:SS                                                            #
#  Copyright (c) Siemens AG, 2026, and licensors. All rights reserved.          #
#  Portions include Open Source Software. See ReadMe_OSS_0.3.html for details.  #
#################################################################################

Environment variable  is not declared.
A log file with more information was generated in the 

```

#### ⚠️ Wrong Workspace

If you provided a workspace path, that is not existing, the following error should be displayed:

```bash
$ roxsie -c example_config.yaml
####################################  ROXSIE  ###################################
#  Version: 0.3.1                                                               #
#  Date:    dd.mm.YYYY                                                          #
#  Time:    HH:MM:SS                                                            #
#  Copyright (c) Siemens AG, 2026, and licensors. All rights reserved.          #
#  Portions include Open Source Software. See ReadMe_OSS_0.3.html for details.  #
#################################################################################

Environment variable path  does not exist.
A log file with more information was generated in the  directory.

```

#### ✔️ Possible Solution

As a solution, the install script `scripts/install_roxsie_environment.sh` can be executed, or the environment variable `USER_ROXSIE_WORKSPACE_PATH` (.env file) can be set manually.

### USER_ROS2_INSTALLATION_PATH

If you are using standard ROS 2 interface configurations e.g. [geometry_msgs/msg/Twist](https://github.com/ros2/common_interfaces/blob/jazzy/geometry_msgs/msg/Twist.msg), the `USER_ROS2_INSTALLATION_PATH` environment variable is crucial for ROXSIE to locate your standard ROS 2 installation directory.\
This variable should point to the root directory of the ROS 2 installation, where all `.msg` and `include/lib` definitions are located.

#### ⚠️ Variable Not Defined

If the `USER_ROS2_INSTALLATION_PATH` environment variable is not defined, ROXSIE will not be able to find necessary standard ROS 2 message definitions, leading to errors during gathering all data type information.\
The following error message will be displayed:

```bash
$ roxsie -c example_config.yaml
####################################  ROXSIE  ###################################
#  Version: 0.3.1                                                               #
#  Date:    dd.mm.YYYY                                                          #
#  Time:    HH:MM:SS                                                            #
#  Copyright (c) Siemens AG, 2026, and licensors. All rights reserved.          #
#  Portions include Open Source Software. See ReadMe_OSS_0.3.html for details.  #
#################################################################################

Gathering ROS 2 message types information ...
Environment variable  is not declared.
A log file with more information was generated in the  directory.

```

#### ⚠️ Wrong ROS 2 Installation Path

If the provided `USER_ROS2_INSTALLATION_PATH` points to a directory, which does not exist, the output should look like the following:

```bash
$ roxsie -c example_config.yaml
####################################  ROXSIE  ###################################
#  Version: 0.3.1                                                               #
#  Date:    dd.mm.YYYY                                                          #
#  Time:    HH:MM:SS                                                            #
#  Copyright (c) Siemens AG, 2026, and licensors. All rights reserved.          #
#  Portions include Open Source Software. See ReadMe_OSS_0.3.html for details.  #
#################################################################################

Gathering ROS 2 message types information ...
Environment variable path  does not exist.
A log file with more information was generated in the  directory.

```

#### ✔️ Possible Solution

You can solve those issues by either running the install script `scripts/install_roxsie_environment.sh` or manually setting the environment variable `USER_ROS2_INSTALLATION_PATH`.\
With the manual step, the `USER_ROS2_INSTALLATION_PATH` variable needs to point to the correct path of your ROS 2 installation. This can be a standard ROS 2 installation like `/opt/ros/jazzy` or a source installation such as `/home/user/jazzy_ws`.\
Note that the naming extension `.../install` is important if you are using a custom ROS 2 installation.

### USER_ROS2_INTERFACE_INSTALLATION_PATH

In addition to standard ROS 2 message types, you can define and provide custom .msg definitions to ROXSIE.\
Therefore, the environment variable `USER_ROS2_INTERFACE_INSTALLATION_PATH` has to be set to provide the path to your custom ROS 2 interface installation directory e.g. `/home/user/ros2_ws`.

#### ⚠️ Type Not Found

If you defined custom ROS 2 .msg definitions and the `USER_ROS2_INTERFACE_INSTALLATION_PATH` environment variable is not defined, ROXSIE will not be able to find the corresponding custom ROS 2 message definitions, leading to errors during the gathering of all data type information.\
An output like the following will be displayed:

```bash
$ roxsie -c custom_config.yaml
####################################  ROXSIE  ###################################
#  Version: 0.3.1                                                               #
#  Date:    dd.mm.YYYY                                                          #
#  Time:    HH:MM:SS                                                            #
#  Copyright (c) Siemens AG, 2026, and licensors. All rights reserved.          #
#  Portions include Open Source Software. See ReadMe_OSS_0.3.html for details.  #
#################################################################################

Gathering ROS 2 message types information ...
Could not find ROS 2 type file . Make sure the ENV variable  is set if you provided a custom ROS 2 interface in the YAML config.
A log file with more information was generated in the  directory.

```

#### ✔️ Possible Solution

You can solve those issues by either running the install script `scripts/install_roxsie_environment.sh` or manually setting the environment variable `USER_ROS2_INTERFACE_INSTALLATION_PATH`.

## Usage Errors

As ROXSIE is a command-line tool, errors can occur easily and therefore, the following common errors can occur.

### Ownership of `start-roxsie` Script

It could occur that the `start-roxsie` script in `/usr/local/bin` has a problem with the ownership when using the binary installer.

```bash
$ start-roxsie
/bin/bash: /usr/local/bin/start-roxsie: Permission denied

```

#### ✔️ Possible Solution

You can fix it changing the ownership of the script:

```bash
sudo chown $USER:$USER /usr/local/bin/start-roxsie

```

### No Configuration File

⚠️ ROXSIE is used with the `--config [-c]` parameter but without the required YAML configuration file name.

```bash
$ roxsie -c
####################################  ROXSIE  ###################################
#  Version: 0.3.1                                                               #
#  Date:    dd.mm.YYYY                                                          #
#  Time:    HH:MM:SS                                                            #
#  Copyright (c) Siemens AG, 2026, and licensors. All rights reserved.          #
#  Portions include Open Source Software. See ReadMe_OSS_0.3.html for details.  #
#################################################################################

Invalid option provided: missing argument for -c

```

✔️ Provide the exact name of the YAML configuration file that your want to use and which is placed in the `input directory` of your ROXSIE workspace.

### Wrong Configuration File Name

⚠️ ROXSIE is used with the `--config [-c]` parameter and a non-existing YAML configuration file.

```bash
$ roxsie -c non_existing_config.yaml
####################################  ROXSIE  ###################################
#  Version: 0.3.1                                                               #
#  Date:    dd.mm.YYYY                                                          #
#  Time:    HH:MM:SS                                                            #
#  Copyright (c) Siemens AG, 2026, and licensors. All rights reserved.          #
#  Portions include Open Source Software. See ReadMe_OSS_0.3.html for details.  #
#################################################################################

YAML configuration file  does not exist in workspace input directory.
A log file with more information was generated in the  directory.

```

✔️ Provide the name of a YAML configuration file name, which file exists in the ROXSIE workspace `input directory`.

### No PSC File

⚠️ ROXSIE is used with a valid YAML configuration file as well as the `--psc_file [-t]` parameter, but without a corresponding .psc file name.

```bash
$ roxsie -c example_config.yaml -t
####################################  ROXSIE  ###################################
#  Version: 0.3.1                                                               #
#  Date:    dd.mm.YYYY                                                          #
#  Time:    HH:MM:SS                                                            #
#  Copyright (c) Siemens AG, 2026, and licensors. All rights reserved.          #
#  Portions include Open Source Software. See ReadMe_OSS_0.3.html for details.  #
#################################################################################

Invalid option provided: missing argument for -t

```

✔️ Provide a .psc file, that exists in the ROXSIE workspace `input directory`.\
Note that this parameter combination only makes sense with the use of the [--non_interactive [-y]](usage/additional_resources/command_line_tool.html#disable-user-input-wizard) flag to go through the whole generation process without user interaction.

### Wrong PSC File Name

⚠️ ROXSIE is used with a valid non-interactive parameter combination but with providing a .psc file name, that either does not exist in the input directory or is spelled wrong.

```bash
$ roxsie -y -c example_config.yaml -t non_existing_project.psc
####################################  ROXSIE  ###################################
#  Version: 0.3.1                                                               #
#  Date:    dd.mm.YYYY                                                          #
#  Time:    HH:MM:SS                                                            #
#  Copyright (c) Siemens AG, 2026, and licensors. All rights reserved.          #
#  Portions include Open Source Software. See ReadMe_OSS_0.3.html for details.  #
#################################################################################

Gathering ROS 2 message types information ...
Generating PLC code ...
TIA project .psc file  does not exist in workspace input directory.
A log file with more information was generated in the  directory.

```

✔️ Provide a .psc file, that exists in the ROXSIE workspace `input directory` or is spelled correct.

### Missing PSC File in Non-Interactive Mode

⚠️ ROXSIE is used in non-interactive mode ([-y](usage/additional_resources/command_line_tool.html#disable-user-input-wizard)), which requires the .psc file an additional argument.

```bash
$ roxsie -c example_config.yaml -y
####################################  ROXSIE  ###################################
#  Version: 0.3.1                                                               #
#  Date:    dd.mm.YYYY                                                          #
#  Time:    HH:MM:SS                                                            #
#  Copyright (c) Siemens AG, 2026, and licensors. All rights reserved.          #
#  Portions include Open Source Software. See ReadMe_OSS_0.3.html for details.  #
#################################################################################

Gathering ROS 2 message types information ...
Generating PLC code ...
The argument --psc_file [t] and .psc file name must be set in non-interactive mode.
A log file with more information was generated in the  directory.

```

✔️ Extend the command with the parameter `--psc_file` or `-t` plus the name of the .psc file you want to use and which is placed in the ROXSIE workspace `input directory`.

### Wrong PSC File For Provided YAML Configuration

⚠️ ROXSIE is used in non-interactive mode with a YAML configuration, that does not match the provided .psc file. Therefore the ROXSIE API code fails to compile.

```bash
$ roxsie -y -c project_1.yaml -t project_2.psc
####################################  ROXSIE  ###################################
#  Version: 0.3.1                                                               #
#  Date:    dd.mm.YYYY                                                          #
#  Time:    HH:MM:SS                                                            #
#  Copyright (c) Siemens AG, 2026, and licensors. All rights reserved.          #
#  Portions include Open Source Software. See ReadMe_OSS_0.3.html for details.  #
#################################################################################

Gathering ROS 2 message types information ...
Generating PLC code ...
Generating ROXSIE API library and ROS 2 package ...
Could not generate ROXSIE API code.
A log file with more information was generated in the  directory.

```

✔️ Provide a .psc file, that was previously generated with the same YAML configuration based PLC code.

### ROXSIE API Code Generation Error

⚠️ During the ROS 2 package generation step, the ROXSIE API code generation fails.

```bash
$ roxsie -y -c project_1.yaml -t project_1.psc
####################################  ROXSIE  ###################################
#  Version: 0.3.1                                                               #
#  Date:    dd.mm.YYYY                                                          #
#  Time:    HH:MM:SS                                                            #
#  Copyright (c) Siemens AG, 2026, and licensors. All rights reserved.          #
#  Portions include Open Source Software. See ReadMe_OSS_0.3.html for details.  #
#################################################################################

Gathering ROS 2 message types information ...
Generating PLC code ...
Generating ROXSIE API library and ROS 2 package ...
Could not generate ROXSIE API code.
A log file with more information was generated in the  directory.

```

✔️ Take a look at the generated `/environment` directory, where a file named `_extended_roxsie_logs.log` provides more detailed information about the actual error.\
Please contact us or your Siemens region in case the error can not be solved directly.

## FAQ

### How can I check which RIB version is installed on my IPC?

✔️ The actual RIB version can be checked via the `RIB_App` executable:

```bash
RIB_App --version

```

### How can I check which Industrial OS is installed on my IPC?

✔️ The Industrial OS version information can be displayed with the following command:

```bash
cat /etc/os-release

```

# Video Tutorial for ROXSIE Setup and Usage

This video serves as a brief tutorial on how to install and use ROXSIE.\
Please refer to the [setup](setup/setup.html) and [usage](usage/usage.html) section for a more detail explanation of the step by step procedure.

# Limitations

## String Size

The RIB only supports fixed size arrays of bytes to store string like texts.\
Therefore, ROS 2 message types that contain strings e.g. `string frame_id` in [std_msgs/msg/Header](https://github.com/ros2/common_interfaces/blob/humble/std_msgs/msg/Header.msg) will translate those strings to a `fixed size` byte array with the size of `254 bytes` (as shown in the picture below).\
⚠️ As the byte array will be available as a string on the PLC side, it is also converted on the PLC side. The PLC based string data type defines the maximum string length of 254 bytes.

## Vector Size

As the RIB does not provide dynamic SHM allocation during runtime, all ROS 2 based `vector` types are currently limited to a `fixed size of 64 elements`.\
⚠️ Additionally, a `valid elements counter` variable will be added to the PLC side, so that the user can define, how many valid elements are stored in the vector, if the number is smaller than 64.

## Shared Memory

As the RIB uses shared memory to access data through Industrial OS as well as the Software-PLC (SW-PLC),\
The actual size of the used shared memory area is therefore limited and has to be conform to the memory limit of the SW-PLC.

An example usage is shown in the picture above, where the two ROS 2 message types [geometry_msgs/msg/Twist](https://github.com/ros2/common_interfaces/blob/humble/geometry_msgs/msg/Twist.msg) and [sensor_msgs/msg/BatteryState](https://github.com/ros2/common_interfaces/blob/humble/sensor_msgs/msg/BatteryState.msg) are used to provide data to the PLC (`ros2_to_plc`) via shared memory.

Those two definitions are combined to one shared memory area during runtime, that shall not exceed the actual SW-PLC memory limitations:

| SW-PLC type      | code memory limit [MB] |
| ---------------- | ---------------------- |
| S7-1505 SP (F)   | 2                      |
| S7-1505 SP T (F) | 3                      |
| S7-1507 S (F)    | 5                      |
| S7-1508 S (F)    | 10                     |
| S7-1508 T (F)    | 12.5                   |

⚠️ Note: As those limitations are defining the total PLC code memory limit, the used ROS 2 .msg typse that needs to be transfered to the PLC or vice versa should not be too big regarding memory layout.

## Number of RIB Clients

The RIB and its participant manager `RIB_App` supports up to 128 client applications.\
But as the generated ROS 2 package uses a precompiled RIB API library, currently only `one ROS 2 client connection` is possible, `combined with one SW-PLC based connection`.

## Number of Data Blocks

The SW-PLC supports in total `up to 20 Cyclic Interrupt Organization Blocks (OBs)` as of TIA Portal v19.\
Additionally, the RIB supports only `up to 16 Consumer / Provider DataBlocks`.\
As each topic in the YAML configuration defines one Cyclic Interrupt OB (ROS 2 -> PLC and vice versa), the user needs to make sure that the provided YAML defines less than 20 topics in total or up to 16 topics for each side (ROS 2 -> PLC and vice versa) as shown in the picture below.

⚠️ The user has to keep in mind that the defined topics and therefore Cyclic Interrupt OBs will be added to already existing ones.

## Topic Rates

At this stage, the RIB connections provided in the YAML config should `not exceed` a specified rate of `1000 Hz`.\
⚠️ The jitter for a defined rate highly depends on the complexity of the provided data type. For example, a [geometry_msgs/msg/Twist](https://github.com/ros2/common_interfaces/blob/humble/geometry_msgs/msg/Twist.msg) is able to reach a stable rate of 1 kHz, but a more complex data type such as [trajectory_msgs/msg/MultiDOFJointTrajectory](https://github.com/ros2/common_interfaces/blob/humble/trajectory_msgs/msg/MultiDOFJointTrajectory.msg) with its vector of strings and nested structs will cause a loss of data at this rate or exceeding the PLC cycle time causing the PLC to stop!\
⚠️ The usage of `strings` and/or `vectors` and/or `nested structs` of them `should be avoided at higher frequencies (>100 Hz)` as their conversion is time consuming!

## Generated ROS 2 Package

The generated C++ based ROS 2 package has the following properties:

- The generated ROSie API C++ code is `not thread safe` (as the RIB connection does not provide thread safe shared memory access).
- The global RIB cycle time for ROS 2 values written to the RIB will be set to the highest of all topic frequencies defined in the provided YAML config file.
- The ROSie API code is based on the `C++17` standard.
- The ROSie generated code is based on `ROS 2 Humble`.
- The ROS 2 data type `wstring` is currently `not supported`, but all other primitive ROS 2 types.

## Type Conversions

The following table shows the available standard data types and their corresponding data types in ROS 2, RIB and TIA.

| ROS 2 data type | RIB data type | TIA data type | Comment                                                         |
| --------------- | ------------- | ------------- | --------------------------------------------------------------- |
| `bool`          | RIB_BOOL      | Bool          |                                                                 |
| `byte`          | RIB_BYTE      | Byte          |                                                                 |
| `char`          | RIB_BYTE      | Char          |                                                                 |
| `float32`       | RIB_FLOAT     | Real          |                                                                 |
| `float64`       | RIB_DOUBLE    | LReal         |                                                                 |
| `int8`          | RIB_INT8      | SInt          |                                                                 |
| `int16`         | RIB_INT16     | Int           |                                                                 |
| `int32`         | RIB_INT32     | DInt          |                                                                 |
| `int64`         | RIB_INT64     | LInt          |                                                                 |
| `uint8`         | RIB_UINT8     | USInt         |                                                                 |
| `uint16`        | RIB_UINT16    | UInt          |                                                                 |
| `uint32`        | RIB_UINT32    | UDInt         |                                                                 |
| `uint64`        | RIB_UINT64    | ULInt         |                                                                 |
| `string`        | RIB_BYTE[254] | String        | The maximum string length is currently capped at 254 characters |
| `wstring`       | unsupported   | unsupported   | Wide strings are not supported yet                              |

For arrays the conversion is:

| ROS 2 data type | RIB data type | TIA data type   | Comment |
| --------------- | ------------- | --------------- | ------- |
| [10]            | [10]          | Array of [0..9] |         |

For vectors the conversion is:

| ROS 2 data type | RIB data type | TIA data type       | Comment                                                                                                                                                                           |
| --------------- | ------------- | ------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| []              | [64]          | Array of [0..63] of | Vectors will be converted to arrays of a fixed size (currently storing 64 elements)                                                                                               |
|                 | RIB_UINT32    | UDInt               | Additionally a `i_valid_elements_*` variable will be created for each ROS 2 vector on the RIB and TIA side to track the actual elements the ROS 2 vector contains / shall contain |

⚠️ Transferring ROS 2 vectors with more elements than the maximum size (current default is 64) will lead to data loss.

# SIMATIC ROS Connector - ROSie

ROSie is a command-line-based code generation tool designed to facilitate the integration of ROS 2 with SIMATIC Software PLC connections based on shared memory (RIB). It operates based on a YAML configuration file to generate the required code.

## Requirements

To utilize ROSie there are two systems involved:

1. **Engineering System**: Used to run the ROSie generator (e.g., an engineering machine with a Linux operating system).
1. **Runtime System**: Used to execute the generated code (e.g., a SIMATIC Industrial OS system with RIB and Software PLC).

### 1. Engineering System Requirements

#### Hardware

- Any x86/x64 based system with at least 1 GB of free disk space

#### Software (all of the listed)

- Linux operating system (e.g. [SIMATIC Industrial OS](https://support.industry.siemens.com/cs/document/109825741/simatic-industrial-os-v3-4-2?dti=0&lc=en-WW), Ubuntu, Debian)
- [ROS 2 Humble](https://docs.ros.org/en/humble/index.html) / [Jazzy](https://docs.ros.org/en/jazzy/index.html)
- [Docker](https://www.docker.com/) (+ Docker Compose plugin) >= v20.x.y
- [TIA Portal](https://www.siemens.com/global/en/products/automation/industry-software/automation-software/tia-portal.html) v19

### 2. Runtime System Requirements

#### Hardware (one of the listed)

- [SIMATIC ET 200SP Open Controller CPU 1515SP PC2](https://mall.industry.siemens.com/mall/en/WW/Catalog/Product/6ES7677-2DB43-0GB1) - [(F)](https://mall.industry.siemens.com/mall/en/WW/Catalog/Product/6ES7677-2SB43-0GB1)
- [SIMATIC IPC 427E](https://mall.industry.siemens.com/mall/en/WW/Catalog/Product/?mlfb=6AG4141-.....-....)
- [SIMATIC IPC BX-39A](https://mall.industry.siemens.com/mall/de/WW/Catalog/Product/?mlfb=6AG4142-.....-....)

All of the listed devices have been tested and verified to work with ROSie generated code.\
This list is non-exhaustive and other devices might work with ROSie too as long as they fulfil the following software requirements.

#### Software (all of the listed)

- [SIMATIC Industrial OS](https://support.industry.siemens.com/cs/document/109825741/simatic-industrial-os-v3-4-2?dti=0&lc=en-WW) = v3.4.2
- [Realtime Information Backbone (RIB)]() = v2.1.1
- [SIMATIC S7-1500 Software Controller]() = v30.1 (CPU 1505SP (F) pre-installed on the [Open Controller](#SIMATIC_ET_200SP_Open_Controller_CPU_1515SP_PC2); [CPU 1507S](https://mall.industry.siemens.com/mall/en/WW/Catalog/Product/6ES7672-7AD02-0YG0) - [(F)](https://mall.industry.siemens.com/mall/en/WW/Catalog/Product/6ES7672-7FD02-0YG0), [CPU 1508S](https://mall.industry.siemens.com/mall/en/WW/Catalog/Product/6ES7672-8AD02-0YG0) - [(F)](https://mall.industry.siemens.com/mall/en/WW/Catalog/Product/6ES7672-8FD02-0YG0) are independent products that need to be bought in addition to an IPC)
- ROS 2 Humble / Jazzy environment to run the generated ROS 2 package (native or dockerized)

## Setup

Detailed setup instructions are available in the [setup instructions](setup.html). The setup process describes the ROSie workspace configuration via a provided installation script.

## Usage

Once the setup is complete, you can enter the ROSie generator environment by executing:

```bash
start-rosie

```

This command launches the `ROSie` container. Inside the container, you can use the `start-rosie` command with the appropriate arguments as described in the [command-line interface guide](usage/command_line_tool.html).

Example command using the `valid_config_example.yaml` file in the `input` directory (see the [ROSie YAML syntax documentation](usage/yaml_config_structure.html)):

```bash
rosie -c valid_config_example.yaml

```

For a detailed walkthrough of the generation process, refer to the [detailed usage guide](usage/usage.html).

## Limitations

⚠️ ROSie relies on the Software PLC and RIB as underlying protocol for data exchange, which impose certain limitations on memory usage, data types, and generated source code. These limitations are detailed in the [limitations section](limitations.html).

## Troubleshooting

For known issues and their resolutions, refer to the [troubleshooting section](troubleshooting.html).

# Setup

Copy the installer in the directory where you want to locate the ROSie generator.

To execute the installer you need to change its rights.

```bash
sudo chmod +x 

```

Run the installer with sudo privileges.

```bash
sudo bash 

```

If you use the standard ROS 2 installation path e.g. `/opt/ros/humble`, it will be detected automatically. Otherwise you will be asked to provide the path to your ROS 2 installation.

```bash
Creating directory rosie_workspace
Verifying archive integrity...  100%   MD5 checksums are OK. All good.
Uncompressing Installer for ROSie version 0-2-0  100%  
Loaded image: cr.siemens.com/ros/projects/ros_oie/ros_plc_connector/deploy:0.2.0
Loaded image: cr.siemens.com/ros/projects/ros_oie/ros_plc_connector/deploy:0-2-0
Found ROS 2 installation at: /opt/ros/humble
Are you using custom ROS 2 message packages? (y/n):

```

Afterwards you will be asked if you are using any custom ROS 2 messages that are not part of the standard ROS 2 installation. If that is not the case answer with n and press enter.

```bash
Are you using custom ROS 2 message packages? (y/n): n

```

If you are using custom messages, answer with y and type in the path to the `install` folder of your custom message packages.

```bash
Are you using custom ROS 2 message packages? (y/n): y
Please provide the path to the install folder of the ROS 2 workspace that contains the built custom messages.
Path to custom message workspace install folder (e.g. ros2_ws/install): /home/user/ros2_ws/install
Found custom message packages at: /home/user/ros2_ws/install

```

The ROSie workspace path will be set automatically to the directory of the 'package' folder.

If the setup was successful, you will see the following message:

```bash
Successfully installed ROSie environment.

```

If you want to move the ROSie generator to another location in the future, please rerun the install binary in the new location.\
After the installation process, the newly created `rosie_workspace` directory can be opened e.g. via [Visual Studio Code](https://code.visualstudio.com/).

The created `rosie_workspace` directory contains the following workspace structure:

Details

```bash
rosie_workspace  # The main workspace directory containing all subfolders of ROSie.
    |-- application_example_ros2_deployment  # Contains helper script and docker setup for providing an example docker based deployment of generated ROS 2 packages including RIB libraries.
    |-- docker_images  # Contains the compressed docker image of the ROSie generator.
    |-- documentation  # Location of the user documentation containing the command line tool, usage workflow and YAML config file structure used to specify which data is exchanged between the PLC and ROS 2.
    |-- input  # Input directory where the YAML configuration file and the .psc files have to be located.
        |-- valid_config_example.yaml  # Example configuration file.
    |-- oss  # Contains the used open source libraries and a OSS license overview file.
    |-- output  # Directory where every generated PLC code and ROS 2 package will be generated by ROSie under the directory name "date + timestamp".
    |-- scripts  # Contains the installation script to set up the ROSie environment on the user's system and to start ROSie.
    |-- template  # Directory with example TIA project files which can be used as templates for the user's own TIA project.
    |-- .env  # Environment configuration file for the ROSie generator. This file will be automatically edited by the installation script and stores the following environment variables:
              # [required] USER_ROS2_INSTALLATION_PATH: Path to the the users ROS 2 installation e.g. /opt/ros/humble.
              # [optional] USER_ROS2_INTERFACE_INSTALLATION_PATH: Path to the users custom interface installation path if custom .msg types are specified within the YAML config.
              # [required] USER_ROSIE_WORKSPACE_PATH: The path to this directory where the `.env` and `docker-compose.yaml` files are located.
    |-- docker-compose.yaml  # Docker Compose setup with predefined user volume mounting -> does not need to be changed.
    |-- README.md  # Entrypoint README for workspace descriptions

```

# Troubleshooting

## Environment Variable Errors

This kind of error occurs, if the user did not setup the host environment properly e.g. via the [setup guideline](overview.html#setup) or manually by setting the host environment variables in the `.env file` accordingly, e.g. like shown below.

```bash
# [REQUIRED]
# The users ROS 2 base installation path share directory that contains all standard interfaces including their .msg files.
# This directory has to be be mounted in the ROSie container to be able to lookup all standard types.
# Default: /opt/ros/humble
USER_ROS2_INSTALLATION_PATH=/opt/ros/humble

# [OPTIONAL]
# The user directory where the optionally needed custom interface is installed, e.g. /home/user/ros2_ws/install
USER_ROS2_INTERFACE_INSTALLATION_PATH=/home/user/ros2_ws/install

# [REQUIRED]
# The current directory where this .env file and the whole provided ROSie workspace is located
USER_ROSIE_WORKSPACE_PATH=/home/user/rosie_ws

```

### USER_ROSIE_WORKSPACE_PATH

The `USER_ROSIE_WORKSPACE_PATH` environment variable defines the path to your actual ROSie workspace. Here, the `input` and `output` directories as well as the `docker-compose.yaml` and `.env` files should be located.

#### ⚠️ Variable Not Defined

If you did not define the ROSie workspace, where the input/output directories are located, the following error will be displayed:

```bash
$ rosie -c example_config.yaml
#################################  ROSie  ##################################
#  Version: 0.2.0                                                          #
#  Date:    21.04.2025                                                     #
#  Time:    15:49:48                                                       #
#  Copyright © Siemens AG, 2025, and licensors. All rights reserved.       #
#  Portions include Open Source Software. See ReadMe_OSS.txt for details.  #
#  The usage for safety-related applications is not intended.              #
############################################################################

Environment variable  is not declared.

```

#### ⚠️ Wrong Workspace

If you provided a workspace path, that is not existing, the following error should be displayed:

```bash
$ rosie -c example_config.yaml
#################################  ROSie  ##################################
#  Version: 0.2.0                                                          #
#  Date:    21.04.2025                                                     #
#  Time:    15:50:24                                                       #
#  Copyright © Siemens AG, 2025, and licensors. All rights reserved.       #
#  Portions include Open Source Software. See ReadMe_OSS.txt for details.  #
#  The usage for safety-related applications is not intended.              #
############################################################################

Environment variable path  does not exist.

```

#### ✔️ Possible Solution

As a solution, the install script `scripts/install_rosie_environment.sh` can be executed, or the environment variable `USER_ROSIE_WORKSPACE_PATH` (.env file) can be set manually.

### USER_ROS2_INSTALLATION_PATH

If you are using standard ROS 2 interface configurations e.g. [geometry_msgs/msg/Twist](https://github.com/ros2/common_interfaces/blob/humble/geometry_msgs/msg/Twist.msg), the `USER_ROS2_INSTALLATION_PATH` environment variable is crucial for ROSie to locate your standard ROS 2 installation directory.\
This variable should point to the root directory of the ROS 2 installation, where all .msg definitions are located.

#### ⚠️ Variable Not Defined

If the `USER_ROS2_INSTALLATION_PATH` environment variable is not defined, ROSie will not be able to find necessary standard ROS 2 message definitions, leading to errors during gathering all data type information.\
The following error message will be displayed:

```bash
$ rosie -c example_config.yaml
#################################  ROSie  ##################################
#  Version: 0.2.0                                                          #
#  Date:    21.04.2025                                                     #
#  Time:    15:51:23                                                       #
#  Copyright © Siemens AG, 2025, and licensors. All rights reserved.       #
#  Portions include Open Source Software. See ReadMe_OSS.txt for details.  #
#  The usage for safety-related applications is not intended.              #
############################################################################

Gathering ROS 2 message types information ...
Environment variable  is not declared.

```

#### ⚠️ Wrong ROS 2 Installation Path

If the provided `USER_ROS2_INSTALLATION_PATH` points to a directory, which does not exist, the output should look like the following:

```bash
$ rosie -c example_config.yaml
#################################  ROSie  ##################################
#  Version: 0.2.0                                                          #
#  Date:    21.04.2025                                                     #
#  Time:    15:52:45                                                       #
#  Copyright © Siemens AG, 2025, and licensors. All rights reserved.       #
#  Portions include Open Source Software. See ReadMe_OSS.txt for details.  #
#  The usage for safety-related applications is not intended.              #
############################################################################

Gathering ROS 2 message types information ...
Environment variable path  does not exist.

```

#### ✔️ Possible Solution

You can solve those issues by either running the install script `scripts/install_rosie_environment.sh` or manually setting the environment variable `USER_ROS2_INSTALLATION_PATH`.\
With the manual step, the `USER_ROS2_INSTALLATION_PATH` variable needs to point to the correct path of your ROS 2 installation. This can be a standard ROS 2 installation like `/opt/ros/humble` or a custom installation such as `/home/user/custom_ros2/install`.\
Note that the naming extension `.../install` is important if you are using a custom ROS 2 installation.

### USER_ROS2_INTERFACE_INSTALLATION_PATH

In addition to standard ROS 2 message types, you can define and provide custom .msg definitions to ROSie.\
Therefore, the environment variable `USER_ROS2_INTERFACE_INSTALLATION_PATH` has to be set to provide the path to your custom ROS 2 interface installation directory e.g. `/home/user/ros2_ws/install`.

#### ⚠️ Type Not Found

If you defined custom ROS 2 .msg definitions and the `USER_ROS2_INTERFACE_INSTALLATION_PATH` environment variable is not defined, ROSie will not be able to find the corresponding custom ROS 2 message definitions, leading to errors during the gathering of all data type information.\
An output like the following will be displayed:

```bash
$ rosie -c custom_config.yaml
#################################  ROSie  ##################################
#  Version: 0.2.0                                                          #
#  Date:    21.04.2025                                                     #
#  Time:    15:53:36                                                       #
#  Copyright © Siemens AG, 2025, and licensors. All rights reserved.       #
#  Portions include Open Source Software. See ReadMe_OSS.txt for details.  #
#  The usage for safety-related applications is not intended.              #
############################################################################

Gathering ROS 2 message types information ...
Could not find ROS 2 type file . Make sure the ENV variable  is set if you provided a custom ROS 2 interface in the YAML config.

```

#### ✔️ Possible Solution

You can solve those issues by either running the install script `scripts/install_rosie_environment.sh` or manually setting the environment variable `USER_ROS2_INTERFACE_INSTALLATION_PATH`.

## Usage Errors

As ROSie is a command line tool, errors can occur easily and therefore, the following common errors can occur.

### Ownership of `start-rosie` Script

It could occur that the `start-rosie` script in `/usr/local/bin` has a problem with the ownership when using the binary installer.

```bash
$ start-rosie
/bin/bash: /usr/local/bin/start-rosie: Permission denied

```

#### ✔️ Possible Solution

You can fix it changing the ownership of the script:

```bash
sudo chown $USER:$USER /usr/local/bin/start-rosie

```

### No Configuration File

⚠️ ROSie is used with the `--config [-c]` parameter but without the required YAML configuration file name.

```bash
$ rosie -c
#################################  ROSie  ##################################
#  Version: 0.2.0                                                          #
#  Date:    21.04.2025                                                     #
#  Time:    15:54:01                                                       #
#  Copyright © Siemens AG, 2025, and licensors. All rights reserved.       #
#  Portions include Open Source Software. See ReadMe_OSS.txt for details.  #
#  The usage for safety-related applications is not intended.              #
############################################################################

Invalid option provided: missing argument for -c

```

✔️ Provide the exact name of the YAML configuration file that your want to use and which is placed in the `input directory` of your ROSie workspace.

### Wrong Configuration File Name

⚠️ ROSie is used with the `--config [-c]` parameter and a non-existing YAML configuration file.

```bash
$ rosie -c non_existing_config.yaml
#################################  ROSie  ##################################
#  Version: 0.2.0                                                          #
#  Date:    21.04.2025                                                     #
#  Time:    15:54:51                                                       #
#  Copyright © Siemens AG, 2025, and licensors. All rights reserved.       #
#  Portions include Open Source Software. See ReadMe_OSS.txt for details.  #
#  The usage for safety-related applications is not intended.              #
############################################################################

YAML configuration file  does not exist in workspace input directory.

```

✔️ Provide the name of a YAML configuration file name, which file exists in the ROSie workspace `input directory`.

### No PSC File

⚠️ ROSie is used with a valid YAML configuration file as well as the `--psc_file [-t]` parameter, but without a corresponding .psc file name.

```bash
$ rosie -c example_config.yaml -t
#################################  ROSie  ##################################
#  Version: 0.2.0                                                          #
#  Date:    21.04.2025                                                     #
#  Time:    15:55:27                                                       #
#  Copyright © Siemens AG, 2025, and licensors. All rights reserved.       #
#  Portions include Open Source Software. See ReadMe_OSS.txt for details.  #
#  The usage for safety-related applications is not intended.              #
############################################################################

Invalid option provided: missing argument for -t

```

✔️ Provide a .psc file, that exists in the ROSie workspace `input directory`.\
Note that this parameter combination only makes sense with the use of the [--non_interactive [-y]](usage/command_line_tool.html#disable-user-input-wizard) flag to go through the whole generation process without user interaction.

### Wrong PSC File Name

⚠️ ROSie is used with a valid non-interactive parameter combination but with providing a .psc file name, that either does not exist in the input directory or is spelled wrong.

```bash
$ rosie -y -c example_config.yaml -t non_existing_project.psc
#################################  ROSie  ##################################
#  Version: 0.2.0                                                          #
#  Date:    21.04.2025                                                     #
#  Time:    15:57:11                                                       #
#  Copyright © Siemens AG, 2025, and licensors. All rights reserved.       #
#  Portions include Open Source Software. See ReadMe_OSS.txt for details.  #
#  The usage for safety-related applications is not intended.              #
############################################################################

TIA project .psc file  does not exist in workspace input directory.

```

✔️ Provide a .psc file, that exists in the ROSie workspace `input directory` or is spelled correct.

### Missing PSC File in Non-Interactive Mode

⚠️ ROSie is used in non-interactive mode ([-y](usage/command_line_tool.html#disable-user-input-wizard)), which requires the .psc file an additional argument.

```bash
$ rosie -c example_config.yaml -y
#################################  ROSie  ##################################
#  Version: 0.2.0                                                          #
#  Date:    21.04.2025                                                     #
#  Time:    15:58:54                                                       #
#  Copyright © Siemens AG, 2025, and licensors. All rights reserved.       #
#  Portions include Open Source Software. See ReadMe_OSS.txt for details.  #
#  The usage for safety-related applications is not intended.              #
############################################################################

The argument --psc_file [t] and .psc file name must be set in non-interactive mode.

```

✔️ Extend the command with the parameter `--psc_file` or `-t` plus the name of the .psc file you want to use and which is placed in the ROSie workspace `input directory`.

### Wrong PSC File For Provided YAML Configuration

⚠️ ROSie is used in non-interactive mode with a YAML configuration, that does not match the provided .psc file. Therefore the ROSie API code fails to compile.

```bash
$ rosie -y -c project_1.yaml -t project_2.psc
#################################  ROSie  ##################################
#  Version: 0.2.0                                                          #
#  Date:    21.04.2025                                                     #
#  Time:    15:59:28                                                       #
#  Copyright © Siemens AG, 2025, and licensors. All rights reserved.       #
#  Portions include Open Source Software. See ReadMe_OSS.txt for details.  #
#  The usage for safety-related applications is not intended.              #
############################################################################

Gathering ROS 2 message types information ...
Generating PLC code ...
Generating ROSie API library and ROS 2 package ...
Could not generate ROSie API code.

```

✔️ Provide a .psc file, that was previously generated with the same YAML configuration based PLC code.

# Command Line Interface

## Required Argument

### YAML Configuration File

The only required parameter is the `--config` or the short version `-c` to provide a predefined YAML configuration file to ROSie for code generation.\
To run ROSie with a predefined YAML and start the whole generation process, run the following command:

```bash
rosie --config file_name.yaml

```

Example console output:

```bash
$ rosie -c example_config.yaml
#################################  ROSie  ##################################
#  Version: 0.2.0                                                          #
#  Date:    21.04.2025                                                     #
#  Time:    15:53:21                                                       #
#  Copyright © Siemens AG, 2025, and licensors. All rights reserved.       #
#  Portions include Open Source Software. See ReadMe_OSS.txt for details.  #
#  The usage for safety-related applications is not intended.              #
############################################################################

Gathering ROS 2 message types information ...
Generating PLC code ...
Please import the generated PLC code into your TIA project, export the TIA project .psc file, and place into the ROSie input folder. 
Please provide the file name of the .psc in the prompt when requested, e.g., 'tia_project.psc' and press ENTER to continue the generation process or 'n' and ENTER to exit the program.

```

## Optional Arguments

### Verify YAML Config File

The `--verify` parameter, short version `-v`, can be set to check, if the provided YAML config meets the requirements.\
To verify, if the provided YAML config matches the required syntax, run the following command:

```bash
rosie --verify --config file_name.yaml

```

Example console output:

```bash
$ rosie -v -c example_config.yaml
#################################  ROSie  ##################################
#  Version: 0.2.0                                                          #
#  Date:    21.04.2025                                                     #
#  Time:    15:54:30                                                       #
#  Copyright © Siemens AG, 2025, and licensors. All rights reserved.       #
#  Portions include Open Source Software. See ReadMe_OSS.txt for details.  #
#  The usage for safety-related applications is not intended.              #
############################################################################

YAML verifying and parsing successful.

```

### Disable User Input Wizard

The `--non_interactive` parameter, short version `-y`, can be used to skip all user interactions and process the whole generation process automatically.\
To start the whole generation process non-interactively, additionally the [--psc_file](#psc-file) argument needs to be provided:

```bash
rosie --non_interactive --config file_name.yaml --psc_file tia_project.psc

```

Example console output:

```bash
$ rosie -y -c example_config.yaml -t example_tia_project.psc
#################################  ROSie  ##################################
#  Version: 0.2.0                                                          #
#  Date:    21.04.2025                                                     #
#  Time:    15:59:17                                                       #
#  Copyright © Siemens AG, 2025, and licensors. All rights reserved.       #
#  Portions include Open Source Software. See ReadMe_OSS.txt for details.  #
#  The usage for safety-related applications is not intended.              #
############################################################################

Gathering ROS 2 message types information ...
Generating PLC code ...
Generating ROSie API library and ROS 2 package ...
Successfully generated ROS 2 package and PLC code.

```

### PSC File

The `--psc_file` parameter, short version `-t`, can be used to process all code generation steps without user interaction.\
It will provide the TIA Portal generated PC System Configuration (.psc) file for internal struct comparison against the PLC based data type definitions.\
⚠️ This argument needs to be combined with the [--non_interactive](#disable-user-input-wizard) argument.

Example usage (creates the same command line output as in the section [Disable User Input Wizard](#disable-user-input-wizard)):

```bash
rosie --non_interactive --config file_name.yaml --psc_file tia_project.psc

```

### Usage Help

Calling ROSie without any arguments or with the `--help` argument, short version `-h`, will print command line usage information:

```bash
rosie --help

```

Example console output:

```bash
$ rosie -h
#################################  ROSie  ##################################
#  Version: 0.2.0                                                          #
#  Date:    21.04.2025                                                     #
#  Time:    16:05:02                                                       #
#  Copyright © Siemens AG, 2025, and licensors. All rights reserved.       #
#  Portions include Open Source Software. See ReadMe_OSS.txt for details.  #
#  The usage for safety-related applications is not intended.              #
############################################################################

Command line parameters: 
  --config [-c]           YAML configuration file name, e.g., 'config.yaml' which defines system specifications (required)
  --help [-h]             Get information about the ROSie usage and CLI parameters
  --non_interactive [-y]  Disable the wizard mode and run all stages without user interaction (optional)
  --psc_file [-t]         TIA project .psc file name, e.g., 'tia_project.psc' (optional)
  --verify [-v]           Verification of the provided YAML configuration file (optional)

```

# Usage

## Prerequisites

Completed [ROSie setup](../overview.html#setup)

## Generating ROS 2 to PLC Connection

### 1. Create interface description file config.yaml

At first a YAML file has to be created, which specifies the interfaces between ROS 2 and the used PLC program according to the rules in the section [YAML Configuration Structure](yaml_config_structure.html).

An working configuration could look like this:

```yaml
# System description of the generated PLC code and ROS 2 package
system:
  ros2:
    package_name: rosie_example_package # Generated ROS 2 package name
    node_name: rosie_node # Generated ROS 2 node name
  rib:
    RIB_App_IPv4: 127.0.0.1 # IPv4 address of the SHM control app: RIB_App
    RIB_App_port: 27567 # Port of the SHM control app: RIB_App

# All topics that subscribe to a ROS 2 topic and write its data to the PLC
ros2_to_plc:
  topics:
    - type: "geometry_msgs/msg/Twist.msg" # The full ROS 2 message type defintion
      ros2_topic: "/cmd_vel" # The ROS 2 topic name, also acts as DataBlock identifier in the PLC
      rate: 200.0 # in [Hz]

# All topics that read data from the PLC and publish it on a ROS 2 topic
plc_to_ros2:
  topics:
    - type: "sensor_msgs/msg/BatteryState.msg" # The full ROS 2 message type defintion
      ros2_topic: "/battery_state" # The ROS 2 topic name, also acts as DataBlock identifier in the PLC
      rate: 10.0 # in [Hz]

```

Save the file inside the input folder in your ROSie workspace

### 2. Start ROSie container

If you completed the [ROSie setup](../overview.html#setup) successfully, you can now start the ROSie container by running the following command in the terminal:

```bash
start-rosie

```

Optional: Manual setup of the ROSie environment and start of the container In case the install script was not executed, the ROSie environment can also be set up manually. Navigate to the installed ROSie package:

```bash
cd path/to/rosie_workspace

```

Make sure the following Environment variables in the `.env` file inside the folder are set according to your ROS 2 environment.

```bash
# [REQUIRED]
# The users ROS 2 base installation path share directory that contains all standard interfaces including their .msg files.
# This directory has to be be mounted in the ROSie container to be able to lookup all standard types.
# Default: /opt/ros/humble
USER_ROS2_INSTALLATION_PATH=/opt/ros/humble

# [OPTIONAL]
# The user directory where the optionally needed custom interface is installed, e.g. /home/user/ros2_ws/install
USER_ROS2_INTERFACE_INSTALLATION_PATH=/home/user/ros2_ws/install

# [REQUIRED]
# The current directory where this .env file and the whole provided ROSie workspace is located
USER_ROSIE_WORKSPACE_PATH=/home/user/rosie_ws

```

Afterwards start ROSie with:

```bash
docker compose run --name ROSie --rm ROSie

```

### 3. Generate PLC code using the ROSie cli tool

In the container environment, start the generation of the PLC code:

```bash
rosie -c config.yaml

```

If your configuration is valid the output should look similar to this:

```bash
$ rosie -c config.yaml
#################################  ROSie  ##################################
#  Version: 0.2.0                                                          #
#  Date:    21.04.2025                                                     #
#  Time:    14:21:38                                                       #
#  Copyright © Siemens AG, 2025, and licensors. All rights reserved.       #
#  Portions include Open Source Software. See ReadMe_OSS.txt for details.  #
#  The usage for safety-related applications is not intended.              #
############################################################################

Gathering ROS 2 message types information ...
Generating PLC code ...
Please import the generated PLC code into your TIA project, export the TIA project .psc file, and place into the ROSie input folder. 
Please provide the file name of the .psc in the prompt when requested, e.g., 'tia_project.psc' and press ENTER to continue the generation process or 'n' and ENTER to exit the program.

```

Inside the output folder of your ROSie workspace a time-stamped folder has been created, that contains the generated PLC code. The path to the generated PLC sources is similar to the following:

```bash
workspace/output/time-stamp/plc_code/_rosie_generated_rib_scl_structs.scl

```

### 4. Move generated PLC code to TIA V19 engineering machine

Now move the generated PLC source file `_rosie_generated_rib_scl_structs.scl` to your TIA engineering machine. If you use a virtual machine you can use shared folders for example.

### 5. Import generated PLC source as external source file

Open the TIA project of your IPC or OpenController that shall establish a connection to ROS 2. Select the Software-Controller of the device and import the previously generated PLC source file into your TIA V19 project.

### 6. Generate blocks from external PLC source file

The imported source file can now be used to generate the PLC program blocks of the ROS 2 \<-> PLC connection:

There will be a warning message informing you that any duplicate blocks will be overwritten. Make sure to have a backup of your project and acknowledge the warning dialog.

Afterwards the following blocks are generated into your TIA project:

And the following UDTs representing the ROS 2 msg types inside your TIA portal:

### 7. Create .psc file of the device containing the ROSie generated blocks

To proceed with the generation of the ROS 2 package of the ROS 2 \<-> PLC connection a .psc file has to be created from your device inside the TIA project.

This can be done by pressing on Project/Memory card file/New/PC system configuration file:

Select the name of the .psc file. Don't use whitespaces inside the filename as this file will be used in a Linux-based OS in another step of the generation process.

Load your OpenController or IPC device into the .psc file by dragging and dropping the device onto the created Memory Card file:

A dialog will open. Press on load to load the device configuration into the .psc file.

### 8. Move .psc file to ROSie input directory

Now copy the created .psc file into the input directory of your ROSie workspace. If you are using a virtual machine you can use shared folders for example.

### 9. Use .psc file to continue ROSie process to generate ROS 2 package

As the .psc file is now located in the ROSie context, you can now proceed with the generation of the ros2 package. The ROSie cli tool should still be at the state of the end of [Step3](#3-generate-plc-code-using-the-rosie-cli-tool). Type in the name of the .psc file you created and press enter.

```bash
$ rosie -c config.yaml
#################################  ROSie  ##################################
#  Version: 0.2.0                                                          #
#  Date:    21.04.2025                                                     #
#  Time:    14:21:38                                                       #
#  Copyright © Siemens AG, 2025, and licensors. All rights reserved.       #
#  Portions include Open Source Software. See ReadMe_OSS.txt for details.  #
#  The usage for safety-related applications is not intended.              #
############################################################################

Gathering ROS 2 message types information ...
Generating PLC code ...
Please import the generated PLC code into your TIA project, export the TIA project .psc file, and place into the ROSie input folder. 
Please provide the file name of the .psc in the prompt when requested, e.g., 'tia_project.psc' and press ENTER to continue the generation process or 'n' and ENTER to exit the program.
plc-system-config.psc

```

If the generation was successful, the output of the cli tool should look similar to this:

```bash
$ rosie -c config.yaml
#################################  ROSie  ##################################
#  Version: 0.2.0                                                          #
#  Date:    21.04.2025                                                     #
#  Time:    14:21:38                                                       #
#  Copyright © Siemens AG, 2025, and licensors. All rights reserved.       #
#  Portions include Open Source Software. See ReadMe_OSS.txt for details.  #
#  The usage for safety-related applications is not intended.              #
############################################################################

Gathering ROS 2 message types information ...
Generating PLC code ...
Please import the generated PLC code into your TIA project, export the TIA project .psc file, and place into the ROSie input folder. 
Please provide the file name of the .psc in the prompt when requested, e.g., 'tia_project.psc' and press ENTER to continue the generation process or 'n' and ENTER to exit the program.
plc-system-config.psc
User Input: plc-system-config.psc
TIA project .psc file  found in input workspace. Continuing generation process ...
Generating ROSie API library and ROS 2 package ...
Successfully generated ROS 2 package and PLC code.

```

### 10. Move generated ROS 2 package to your desired workspace and build the package

Congratulations the generation process is now done. You can find the generated ROS 2 package in the output folder of your ROSie workspace under:

```bash
workspace/time-stamp/output/ros2_package/ros2_package_name

```

#### a. Working directly in Industrial OS

If you are working directly in the Industrial OS, simply copy this package to your ROS 2 workspace. Install the RIB libraries on your Industrial OS by running the `install_rib_support_libs.sh` script in the 3rd_party folder:

```bash
cd 3rd_party
sudo bash install_rib_support_libs.sh

```

Afterwards you can build the ROSie generated ROS 2 package.

#### b. Working in another IPC

If you are working in another ROS 2 machine you could create a docker image with the generated ROS 2 node automatically to copy it in the Industrial OS. For that, copy the previously generated ROS 2 package - and in case of custom messages also ROS 2 packages containing the custom message definitions - to the `ros2_ws/src` directory in the `application_example_ros2_deployment` folder:

```bash
workspace/application_example_ros2_deployment/ros2_ws/src

```

Then, navigate to the package folder and run the following command to build the application example:

```bash
cd application_example_ros2_deployment
bash build_application_example.sh

```

The script will first search for the ROSie generated ROS 2 packages in the `ros2_ws/src` directory. If there are multiple ROSie generated ROS 2 packages the script will ask you to select the one you want to deploy.

```bash
Found multiple ROSie generated packages:                                                                                                                
1) Package name: rosie_example_package
   Node name: rosie_node
   Version: 0.2.0
2) Package name: test_package
   Node name: test_node
   Version: 0.2.0
Please select an option: 1

```

After selecting the package, the script will build a docker image based on the ROS 2 Humble docker image, compiling the ROS 2 packages in the `ros2_ws/src` directory and setting up the required runtime dependencies for the generated ROSie node. Afterwards, the `docker-compose.yaml` file will be updated to use the selected package.

Save the docker image.

```bash
docker save rosie_app_example > rosie_app_example.tar

```

Copy the docker image into the Industrial OS.

```bash
scp rosie_app_example.tar [USER]@[DESTINATION_HOST_IP]:rosie_app_example.tar

```

Copy the docker-compose.yaml file into the Industrial OS.

```bash
scp docker-compose.yaml [USER]@[DESTINATION_HOST_IP]:docker-compose.yaml

```

Load the image again in the Industrial OS.

```bash
ssh [USER]@[DESTINATION_HOST_IP]
docker load < rosie_app_example.tar 

```

Now you are ready to run the generated ROS 2 package in the Industrial OS.

## Running the Generated Interface

### 1. Load PLC program to the SW-PLC

If not done already load the TIA project that was extended by the ROSie generated program blocks to the SW-PLC. The PLC-side will try to automatically connect to the RIB_App.

### 2. Start RIB_App

Start the RIB_App inside the Industrial OS of your selected device.

```bash
RIB_App

```

### 3. Start ROS 2 Node

#### a. Working directly in Industrial OS

Start the ROS 2 node by running (make sure the workspace was sourced before):

```bash
ros2 run  

```

#### b. Working in another IPC

Start the ROS 2 node using the `docker-compose.yaml` file in your Industrial OS:

```bash
ssh [USER]@[DESTINATION_HOST_IP]
docker compose up

```

# YAML Configuration Structure

## Introduction

As ROSie provides a command line based interface for users, a service was needed to give users the possibility to specify their system using a [YAML](https://yaml.org) based configuration file.\
In this configuration, users can declare the [system](#system), which stands for the ROS 2 package that will be the wrapper for all defined data structures. Additionally, users shall define the [data structure connections](#topic) based on ROS 2 topics.

## System

An example of a system definition is shown below.\
The generated package will setup a fully functional ROS 2 package, that works out out of the box by using the parameters specified below.

```yaml
system:
  ros2:
    package_name: example_package
    node_name: example_node
  rib:
    RIB_App_IPv4: 127.0.0.1
    RIB_App_port: 27567
  # optional
  plc:
    # optional
    rib_config_db_name: "rib_config_topic_DB"
    rib_connect_state_machine_fc_name: "rib_connect_build_up_FC"

```

### `system (required)`

The `system` keyword specifies the content of the generated ROS 2 package.

```yaml
system:

```

### `ros2 (required)`

The `ros2` keyword declares all content that is ROS 2 specific.

```yaml
  ros2:

```

### `package_name (required)`

The `package_name` keyword is used to define the name for the generated ROS 2 package.

```yaml
    package_name: 

```

### `node_name (required)`

The `node_name` keyword is used to define the name for the generated ROS 2 node inside the package.

```yaml
    node_name: 

```

### `rib (required)`

The `rib` keyword defines the connection to the RIB_App, that is used to manage different shared memory connections of the RIB.

```yaml
  rib:

```

### `RIB_App_IPv4 (required)`

The `RIB_App_IPv4` keyword specifies the IPv4 address of the RIB_App, to which the RIB API will be connected to via TCP based connection.\
Valid values are either `localhost` or a correctly set IPv4 address e.g. `127.0.0.1`.\
Note: The value can later be changed in the source code.

```yaml
    RIB_App_IPv4: localhost

```

### `RIB_App_port (required)`

The `RIB_App_port` keyword is used to define the TCP based port, at which the RIB_App is reachable.\
Valid are full numbers e.g. `80`.\
Note: The value can later be changed in the source code.

```yaml
    RIB_App_port: 

```

### `plc (optional)`

The `plc` keyword declares all content that is PLC specific.

```yaml
  plc:

```

### `rib_config_db_name (optional)`

The `rib_config_db_name` keyword can be used to change the name of the data block associated to the connection control and status of the RIB. The default value is "RIB_CONFIG_DB".

```yaml
    rib_config_db_name: 

```

### `rib_connect_state_machine_fc_name (optional)`

The `rib_connect_state_machine_fc_name` keyword can be used to change the name of the function block handling the automatic connection build up to the RIB. The default value is "RIB_CONNECT_STATE_MACHINE_FC".

```yaml
    rib_connect_state_machine_fc_name: 

```

## Topic

The `Topic` area relies on the principles of the [ROS 2 topic system](https://docs.ros.org/en/humble/Tutorials/Beginner-CLI-Tools/Understanding-ROS2-Topics/Understanding-ROS2-Topics.html) and is divided into two possible connection directions, either from [ROS 2 to PLC](#ros-2-to-plc) or from [PLC to ROS 2](#plc-to-ros-2).\
Those connections are based on the data structures that are defined within them.

### ROS 2 to PLC

To define a ROS 2 subscriber that writes its received data to the PLC, the `ros2_to_plc` tag shall be used.\
Within it, all needed data structures can be specified. Each type specification will be used to create a corresponding subscriber within the generated ROS 2 package.\
An example is provided below.

```yaml
ros2_to_plc:
  topics:
    - type: "custom_msgs/msg/Type.msg"
      ros2_topic: "/topic"
      rate: 100.0
    - type: "custom_msgs/msg/AnotherType.msg"
      ros2_topic: "/another_topic"
      rate: 5.0

```

#### `ros2_to_plc (required)`

The `ros2_to_plc` keyword defines the connection direction from ROS 2 to the PLC.

```yaml
ros2_to_plc:

```

#### `topics (required)`

The `topics` keyword is used to clarify, that the created ROS 2 part will be a subscriber, that listens on a specified topic.

```yaml
  topics:

```

#### `type (required)`

The `type` keyword defines the ROS 2 based topic message type, that will be searched for either in the provided custom interface path or the ROS 2 standard installation path.

```yaml
    - type: "/msg/.msg"

```

✔️ Valid values are:

- In case of a standard ROS 2 based message type e.g. geometry_msgs/msg/Twist located at the path specified via `USER_ROS2_INSTALLATION_PATH` ENV variable:\
  `"geometry_msgs/msg/Twist.msg"`
- In case of a custom message type from a custom interface package located at the path specified via `USER_ROS2_INTERFACE_INSTALLATION_PATH` ENV variable:\
  `"custom_msgs/msg/Type.msg"`

❌ Wrong values are e.g.:

- `"geometry_msgs/msg/Twist"`
- `"geometry_msgs/Twist"`

#### `ros2_topic (required)`

The `ros2_topic` keyword defines the ROS 2 topic name of the desired message type, where the '/' is optional.\
⚠️ The name must be unique in the YAML configuration!

```yaml
      ros2_topic: "/"

```

ROS 2 topic namespaces can be provided e.g.:

```yaml
      ros2_topic: "robot_1/topic_name"

```

#### `rate (required)`

The `rate` keyword is used to define the rate in Hz, in which messages will be written to the PLC on the specified topic.\
It is used to configure the interrupt cycle of the PLC side for this topic.

```yaml
      rate: 10.0 # in Hz

```

✔️ Valid subscription rates [Hz] are (as floating point values):

- All values > 0.0
- All values \<= 1000.0

⚠️ Values must not match the exact real subscription rate, but one has to keep in mind, that each generated PLC interrupt OB will provide a cycle time with the same rate as specified in the YAML!

### PLC to ROS 2

To define a ROS 2 publisher that reads the data received from the PLC and publishes it to ROS 2, the `plc_to_ros2` tag shall be used.\
Within it, all needed data structures can be specified similar to the configurations in the section [ROS 2 to PLC](#ros-2-to-plc). Each type specification will be used to create a corresponding publisher within the generated ROS 2 package.\
An example is provided below.

```yaml
plc_to_ros2:
  topics:
    - type: "custom_msgs/msg/Type.msg"
      ros2_topic: "/topic"
      rate: 250.0
    - type: "sensor_msgs/msg/BatteryState.msg"
      ros2_topic: "/battery"
      rate: 1.0

```

#### `plc_to_ros2 (required)`

The `plc_to_ros2` keyword defines the connection direction from the PLC to ROS 2.

```yaml
plc_to_ros2:

```

#### `topics (required)`

The `topics` keyword is used to clarify, that the created ROS 2 part will be a publisher, that publishes PLC data on a specified topic.

```yaml
  topics:

```

#### `type (required)`

The `type` keyword defines the ROS 2 based topic message type, that will be searched for either in the provided custom interface path or the ROS 2 standard installation path.

```yaml
    - type: "/msg/.msg"

```

✔️ Valid values are:

- In case of a standard ROS 2 based message type e.g. geometry_msgs/msg/Twist located at the path specified via `USER_ROS2_INSTALLATION_PATH` ENV variable:\
  `"geometry_msgs/msg/Twist.msg"`
- In case of a custom message type from a custom interface package located at the path specified via `USER_ROS2_INTERFACE_INSTALLATION_PATH` ENV variable:\
  `"custom_msgs/msg/Type.msg"`

❌ Wrong values are e.g.:

- `"geometry_msgs/msg/Twist"`
- `"geometry_msgs/Twist"`

#### `ros2_topic (required)`

The `ros2_topic` keyword defines the ROS 2 topic name of the desired message type, where the '/' is optional.\
⚠️ The name must be unique in the YAML configuration!

```yaml
      ros2_topic: "/"

```

ROS 2 topic namespaces can be provided e.g.:

```yaml
      ros2_topic: "robot_1/topic_name"

```

#### `rate (required)`

The `rate` keyword is used to define the rate in Hz, in which messages will be read from the PLC on the specified topic.\
It is used to configure the interrupt cycle of the PLC side as well as a ROS 2 based timer callback to read the data at the provided rate.

```yaml
      rate: 10.0 # in Hz

```

✔️ Valid subscription rates [Hz] are (as floating point values):

- All values > 0.0
- All values \<= 1000.0

⚠️ Values must not match the exact real subscription rate, but one has to keep in mind, that each generated PLC interrupt OB will provide a cycle time with the same rate as specified in the YAML!

## Example

Based on provided message type definitions in a custom `interface_package`, a ROSie YAML configuration `config.yaml` could look like the following:

```yaml
system:
  ros2:
    package_name: custom_package
    node_name: custom_node
  rib:
    RIB_App_IPv4: 127.0.0.1
    RIB_App_port: 27567

ros2_to_plc:
  topics:
    - type: "interface_package/msg/Type.msg"
      ros2_topic: "/topic"
      rate: 50.0
    - type: "interface_package/msg/AnotherType.msg"
      ros2_topic: "/another_topic"
      rate: 10.0
    - type: "gemetry_msgs/msg/Twist.msg"
      ros2_topic: "cmd_vel"
      rate: 100.0

plc_to_ros2:
  topics:
    - type: "interface_package/msg/Type2.msg"
      ros2_topic: "namespaced/topic2"
      rate: 250.0
    - type: "geometry_msgs/msg/PoseStamped.msg"
      ros2_topic: "/pose"
      rate: 50.0

```

# Limitations

## String Size

The RIB only supports fixed size arrays of bytes to store string like texts.\
Therefore, ROS 2 message types that contain strings e.g. `string frame_id` in [std_msgs/msg/Header](https://github.com/ros2/common_interfaces/blob/humble/std_msgs/msg/Header.msg) will translate those strings to a `fixed size` byte array with the size of `254 bytes` (as shown in the picture below).\
⚠️ As the byte array will be available as a string on the PLC side, it is also converted on the PLC side. The PLC based string data type defines the maximum string length of 254 bytes.

## Vector Size

As the RIB does not provide dynamic SHM allocation during runtime, all ROS 2 based `vector` types are currently limited to a `fixed size of 64 elements`.\
⚠️ Additionally, a `valid elements counter` variable will be added to the PLC side, so that the user can define, how many valid elements are stored in the vector, if the number is smaller than 64.

## Shared Memory

As the RIB uses shared memory to access data through Industrial OS as well as the Software-PLC (SW-PLC),\
The actual size of the used shared memory area is therefore limited and has to be conform to the memory limit of the SW-PLC.

An example usage is shown in the picture above, where the two ROS 2 message types [geometry_msgs/msg/Twist](https://github.com/ros2/common_interfaces/blob/humble/geometry_msgs/msg/Twist.msg) and [sensor_msgs/msg/BatteryState](https://github.com/ros2/common_interfaces/blob/humble/sensor_msgs/msg/BatteryState.msg) are used to provide data to the PLC (`ros2_to_plc`) via shared memory.

Those two definitions are combined to one shared memory area during runtime, that shall not exceed the actual SW-PLC memory limitations:

| SW-PLC type      | Code memory limit [MB] |
| ---------------- | ---------------------- |
| S7-1505 SP (F)   | 2                      |
| S7-1505 SP T (F) | 3                      |
| S7-1507 S (F)    | 5                      |
| S7-1508 S (F)    | 10                     |
| S7-1508 T (F)    | 12.5                   |

⚠️ Note: As those limitations are defining the total PLC code memory limit, the used ROS 2 .msg typse that needs to be transfered to the PLC or vice versa should not be too big regarding memory layout.

## Number of RIB Clients

The RIB and its participant manager `RIB_App` supports up to 128 client applications.\
But as the generated ROS 2 package uses a precompiled RIB API library, currently only `one ROS 2 client connection` is possible, `combined with one SW-PLC based connection`.

## Number of Data Blocks / Symbols

The SW-PLC supports in total `up to 20 Cyclic Organization Blocks (OBs)`.\
As each topic in the YAML configuration defines one Cyclic Interrupt OB (ROS 2 -> PLC and vice versa), the user needs to make sure that the provided YAML defines less than 20 topics in total or up to 16 topics for each side (ROS 2 -> PLC and vice versa) as shown in the picture below.

⚠️ The user has to keep in mind that the defined topics and therefore Cyclic Interrupt OBs will be added to already existing ones.

Additionally, the RIB provides the following maximum specifications:

| RIB      | Max. number of symbols (topics) | Max. number of provide / consume DBs |
| -------- | ------------------------------- | ------------------------------------ |
| `v2.1.1` | 1024                            | 16                                   |
| `v2.2.4` | 8192                            | 1024                                 |

## Topic Rates

At this stage, the RIB connections provided in the YAML config should `not exceed` a specified rate of `1000 Hz`.\
⚠️ The jitter for a defined rate highly depends on the complexity of the provided data type. For example, a [geometry_msgs/msg/Twist](https://github.com/ros2/common_interfaces/blob/humble/geometry_msgs/msg/Twist.msg) is able to reach a stable rate of 1 kHz, but a more complex data type such as [trajectory_msgs/msg/MultiDOFJointTrajectory](https://github.com/ros2/common_interfaces/blob/humble/trajectory_msgs/msg/MultiDOFJointTrajectory.msg) with its vector of strings and nested structs will cause a loss of data at this rate or exceeding the PLC cycle time causing the PLC to stop!\
⚠️ The usage of `strings` and/or `vectors` and/or `nested structs` of them `should be avoided at higher frequencies (>100 Hz)` as their conversion is time consuming!

## Generated ROS 2 Package

The generated C++ based ROS 2 package has the following properties:

- The generated ROSie API C++ code is `not thread safe` (as the RIB connection does not provide thread safe shared memory access).
- The global RIB cycle time for ROS 2 values written to the RIB will be set to the highest of all topic frequencies defined in the provided YAML config file.
- The ROSie API code is based on the `C++17` standard.
- The ROSie generated ROS 2 code is only tested with `Humble` and `Jazzy`.
- The ROS 2 data type `wstring` is currently `not supported`, but all other primitive ROS 2 types.

## Type Conversions

The following table shows the available standard data types and their corresponding data types in ROS 2, RIB and TIA.

| ROS 2 data type | RIB data type | TIA data type | Comment                                                         |
| --------------- | ------------- | ------------- | --------------------------------------------------------------- |
| `bool`          | RIB_BOOL      | Bool          |                                                                 |
| `byte`          | RIB_BYTE      | Byte          |                                                                 |
| `char`          | RIB_BYTE      | Char          |                                                                 |
| `float32`       | RIB_FLOAT     | Real          |                                                                 |
| `float64`       | RIB_DOUBLE    | LReal         |                                                                 |
| `int8`          | RIB_INT8      | SInt          |                                                                 |
| `int16`         | RIB_INT16     | Int           |                                                                 |
| `int32`         | RIB_INT32     | DInt          |                                                                 |
| `int64`         | RIB_INT64     | LInt          |                                                                 |
| `uint8`         | RIB_UINT8     | USInt         |                                                                 |
| `uint16`        | RIB_UINT16    | UInt          |                                                                 |
| `uint32`        | RIB_UINT32    | UDInt         |                                                                 |
| `uint64`        | RIB_UINT64    | ULInt         |                                                                 |
| `string`        | RIB_BYTE[254] | String        | The maximum string length is currently capped at 254 characters |
| `wstring`       | unsupported   | unsupported   | Wide strings are not supported yet                              |

For arrays the conversion is:

| ROS 2 data type | RIB data type | TIA data type       | Comment                                                                                                                                                                           |
| --------------- | ------------- | ------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| []              | [64]          | Array of [0..63] of | Vectors will be converted to arrays of a fixed size (currently storing 64 elements)                                                                                               |
|                 | RIB_UINT32    | UDInt               | Additionally a `i_valid_elements_*` variable will be created for each ROS 2 vector on the RIB and TIA side to track the actual elements the ROS 2 vector contains / shall contain |

⚠️ Transferring ROS 2 vectors with more elements than the maximum size (current default is 64) will lead to data loss.

# SIMATIC ROS Connector - ROSie

[ROSie](https://siemens.com/ros) is a command-line-based code generation tool designed to facilitate the integration of ROS 2 with SIMATIC Software PLC connections based on shared memory. It operates based on a YAML configuration file to generate the required PLC and ROS 2 code.

✔️ It allows you to have a connection between your ROS 2 runtime and your PLC system, which are both coexisting, running and sharing information among each other inside the same Siemens industrial hardware, combining the [advantages of both worlds](ros_plc_overview.html#integrating-plcs-with-ros)!

The process to integrate the communication between the systems consist of two steps:

**1. Generate SCL Code:** Generator creates a SCL source file that needs to be imported into your TIA-Portal project. It contains all program blocks required for the data exchange with ROS 2.\
**2. Generate ROS 2 package:** A fully functional ROS 2 package is generated, which contains the connection to the Software PLC.

After the generation process, both programs can be executed on [suitable devices](requirements.html#2-runtime-system-requirements).

# Requirements

To utilize ROSie there are two systems involved:

1. **Engineering System**: Used to run the ROSie generator (e.g., a Windows engineering machine with Linux as subsystem).
1. **Runtime System**: Used to execute the generated code (e.g., a SIMATIC Industrial OS system with RIB and Software PLC).

## 1. Engineering System Requirements

### Hardware

- Any x86/x64 based system with at least 1 GB of free disk space

### Software (all of the listed)

- [ROSie](https://siemens.com/ros) container running in a Linux environment with access to ROS 2 installation via [ENV variables](usage/usage.html#2-start-rosie-container)
- Linux (e.g. [SIMATIC Industrial OS](https://mall.industry.siemens.com/mall/en/WW/Catalog/Products/10355854?activeTab=productinformation), Ubuntu, Debian) + Windows 10/11 (for TIA Portal) operating systems
- [ROS 2](https://github.com/ros2) running natively in Linux or in a container if installation is mounted to host to be accessible by ROSie container
- [Docker](https://www.docker.com/) (+ Docker Compose plugin) `>= v20.x.y`
- [TIA Portal](https://www.siemens.com/global/en/products/automation/industry-software/automation-software/tia-portal.html)

⚠️ **Note:** To be able to use the RIB with Software Controller `v31.1`, TIA Portal v20 `>= Update 1` needs to be installed.

The use of virtual machines or [Windows Subsystem for Linux (WSL)](https://learn.microsoft.com/en-us/windows/wsl/install) is recommended in order to have TIA Portal and ROS 2 running in parallel in the same system.\
Please refer to the [Engineering System Setup](setup/engineering_system.html) for a detailed step by step instruction on how to prepare your engineering system.

## 2. Runtime System Requirements

### Hardware (one of the listed)

- [SIMATIC ET 200SP Open Controller CPU 1515SP PC2](https://mall.industry.siemens.com/mall/en/WW/Catalog/Product/6ES7677-2DB43-0GB1) - [(F)](https://mall.industry.siemens.com/mall/en/WW/Catalog/Product/6ES7677-2SB43-0GB1)
- [SIMATIC IPC 427E](https://mall.industry.siemens.com/mall/en/WW/Catalog/Product/?mlfb=6AG4141-.....-....)
- [SIMATIC IPC BX-39A](https://mall.industry.siemens.com/mall/de/WW/Catalog/Product/?mlfb=6AG4142-.....-....)

All of the listed devices have been tested and verified to work with ROSie generated code.\
This list is non-exhaustive and other devices might work with ROSie too as long as they fulfil the following software requirements.

### Software (all of the listed)

- [SIMATIC Industrial OS](https://mall.industry.siemens.com/mall/en/WW/Catalog/Products/10355854?activeTab=productinformation)
- [Realtime Information Backbone (RIB)]()
- [SIMATIC S7-1500 Software Controller (PLC)](https://www.siemens.com/global/en/products/automation/systems/industrial/plc/simatic-software-controller.html) (CPU 1505SP (F) pre-installed on the [Open Controller](#SIMATIC_ET_200SP_Open_Controller_CPU_1515SP_PC2); [CPU 1507S](https://mall.industry.siemens.com/mall/en/WW/Catalog/Product/6ES7672-7AD02-0YG0) - [(F)](https://mall.industry.siemens.com/mall/en/WW/Catalog/Product/6ES7672-7FD02-0YG0), [CPU 1508S](https://mall.industry.siemens.com/mall/en/WW/Catalog/Product/6ES7672-8AD02-0YG0) - [(F)](https://mall.industry.siemens.com/mall/en/WW/Catalog/Product/6ES7672-8FD02-0YG0) are independent products that need to be bought in addition to an IPC)
- [ROS 2](https://github.com/ros2) installation to run the generated ROS 2 package (native or via container)

## Software Version Compatibilities

The following table shows the supported software versions of all ROSie releases:

| ROSie    | Software Controller                                                                                                                                                                                                                                                                                                                               | TIA Portal                                                                                                                                                                                                                                                                                                                                                                | Industrial OS                                                                                                                                                                                        | RIB            | ROS 2 (tested)                                                                                         |
| -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------- | ------------------------------------------------------------------------------------------------------ |
| `v0.1.0` | [v30.1]()                                                                                                                                                                       | [v19](https://support.industry.siemens.com/cs/document/109820994/simatic-step-7-inkl-safety-s7-plcsim-and-wincc-v19-trial-download)                                                                                                                                                                                                                                       | [v3.4.2](https://support.industry.siemens.com/cs/document/109825741/simatic-industrial-os-v3-4-2)                                                                                                    | v2.1.1         | [Humble](https://docs.ros.org/en/humble/index.html)                                                    |
| `v0.1.1` | [v30.1]()                                                                                                                                                                       | [v19](https://support.industry.siemens.com/cs/document/109820994/simatic-step-7-inkl-safety-s7-plcsim-and-wincc-v19-trial-download)                                                                                                                                                                                                                                       | [v3.4.2](https://support.industry.siemens.com/cs/document/109825741/simatic-industrial-os-v3-4-2)                                                                                                    | v2.1.1         | [Humble](https://docs.ros.org/en/humble/index.html)                                                    |
| `v0.2.0` | [v30.1]()                                                                                                                                                                       | [v19](https://support.industry.siemens.com/cs/document/109820994/simatic-step-7-inkl-safety-s7-plcsim-and-wincc-v19-trial-download)                                                                                                                                                                                                                                       | [v3.4.2](https://support.industry.siemens.com/cs/document/109825741/simatic-industrial-os-v3-4-2)                                                                                                    | v2.1.1         | [Humble](https://docs.ros.org/en/humble/index.html), [Jazzy](https://docs.ros.org/en/jazzy/index.html) |
| `v0.3.0` | [v30.1](), [v31.1]() | [v19](https://support.industry.siemens.com/cs/document/109820994/simatic-step-7-inkl-safety-s7-plcsim-and-wincc-v19-trial-download), [v20](https://support.industry.siemens.com/cs/document/109963850/simatic-step-7-inkl-safety-s7-plcsim-und-wincc-v20-trial-download) >= [Update 1](https://support.industry.siemens.com/cs/document/109963851/tia-portal-v20-updates) | [v3.4.2](https://support.industry.siemens.com/cs/document/109825741/simatic-industrial-os-v3-4-2), [v3.5.3](https://support.industry.siemens.com/cs/document/109983416/simatic-industrial-os-v3-5-3) | v2.1.1, v2.2.4 | [Humble](https://docs.ros.org/en/humble/index.html), [Jazzy](https://docs.ros.org/en/jazzy/index.html) |

Additionally, each Software Controller version requires a specific TIA Portal, Industrial OS and RIB version:

| Software Controller                                                                                                                                                           | TIA Portal                                                                                                                                                                                                                           | Industrial OS                                                                                     | RIB    |
| ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------- | ------ |
| [`v30.1`]() | [v19](https://support.industry.siemens.com/cs/document/109820994/simatic-step-7-inkl-safety-s7-plcsim-and-wincc-v19-trial-download)                                                                                                  | [v3.4.2](https://support.industry.siemens.com/cs/document/109825741/simatic-industrial-os-v3-4-2) | v2.1.1 |
| [`v31.1`]()        | [v20](https://support.industry.siemens.com/cs/document/109963850/simatic-step-7-inkl-safety-s7-plcsim-und-wincc-v20-trial-download) >= [Update 1](https://support.industry.siemens.com/cs/document/109963851/tia-portal-v20-updates) | [v3.5.3](https://support.industry.siemens.com/cs/document/109983416/simatic-industrial-os-v3-5-3) | v2.2.4 |

# Overview of ROS and PLC Technology

## Programmable Logic Controller (PLC)

[Programmable Logic Controllers (PLCs)](https://www.siemens.com/global/en/products/automation/systems/industrial/plc.html) are robust industrial computers specifically designed for automating manufacturing processes. They are widely deployed in factory automation, industrial control systems, and production lines where reliability and consistent performance are critical.\
PLCs incorporate robust safety features including failsafe operation, safety-certified hardware variants, and compliance with industrial safety standards (IEC 61508/61131).

- **Key Features:**
  - Real-time, deterministic operation
  - High reliability and fault tolerance
  - Designed for 24/7 operation in harsh environments
  - Robust safety features
- **Programming:** Ladder Logic, Structured Text, Function Block Diagram
- **Strengths:** Reliability, stability, industrial standards compliance
- **Limitations:** Limited computational power, specialized programming knowledge

## Robot Operating System (ROS)

The [Robot Operating System (ROS)](https://www.ros.org) is a flexible framework and middleware that facilitates robot software development. It has become the standard platform for research robots, autonomous vehicles, and service robots where adaptability and access to advanced capabilities are required.

- **Key Features:**
  - Distributed computing architecture
  - Rich ecosystem of tools and packages
  - Hardware abstraction capabilities
- **Programming:** Python, C++, with node-based communication model
- **Strengths:** Flexibility, extensive libraries, active open-source community
- **Limitations:** Not inherently real-time, complexity, steeper learning curve

## Integrating PLCs with ROS

Integrating PLCs with ROS creates a powerful synergy that leverages the strengths of both technologies.\
PLCs contribute industrial-grade reliability, deterministic operation, and safety certification essential for critical manufacturing processes, while ROS provides advanced capabilities in motion planning, perception, and AI and gives access to latest research results.

This combination enables sophisticated robotic applications that maintain industrial robustness - allowing manufacturers to implement complex automation solutions like adaptive manufacturing, intelligent material handling, and flexible production systems without sacrificing the reliability demanded by industrial environments.

# Troubleshooting

## Environment Variable Errors

This kind of error occurs, if the user did not setup the host environment properly e.g. via the [setup guideline](setup/setup.html) or manually by setting the host environment variables in the `.env file` accordingly, e.g. like shown below.

```bash
# [REQUIRED]
# The users ROS 2 base installation path share directory that contains all standard interfaces including their .msg files.
# This directory has to be be mounted in the ROSie container to be able to lookup all standard types.
# Standard ROS 2 installation example: /opt/ros/jazzy
# Custom ROS 2 installation example:   /home/user/jazzy_ws
USER_ROS2_INSTALLATION_PATH=/opt/ros/jazzy

# [OPTIONAL]
# The user directory where the optionally needed custom ROS 2 interfaces are installed
# Example: /home/user/ros2_ws
USER_ROS2_INTERFACE_INSTALLATION_PATH=/home/user/ros2_ws

# [REQUIRED]
# The current directory where this .env file and the whole provided ROSie workspace is located
USER_ROSIE_WORKSPACE_PATH=/home/user/rosie_workspace

```

### USER_ROSIE_WORKSPACE_PATH

The `USER_ROSIE_WORKSPACE_PATH` environment variable defines the path to your actual ROSie workspace. Here, the `input` and `output` directories as well as the `docker-compose.yaml` and `.env` files should be located.

#### ⚠️ Variable Not Defined

If you did not define the ROSie workspace, where the input/output directories are located, the following error will be displayed:

```bash
$ rosie -c example_config.yaml
#####################################  ROSie  #####################################
#  Version: 0.3.0                                                                 #
#  Date:    08.07.2025                                                            #
#  Time:    15:49:48                                                              #
#  Copyright (c) Siemens AG, 2025, and licensors. All rights reserved.            #
#  Portions include Open Source Software. See ReadMe_OSS_0.3.0.html for details.  #
###################################################################################

Environment variable  is not declared.

```

#### ⚠️ Wrong Workspace

If you provided a workspace path, that is not existing, the following error should be displayed:

```bash
$ rosie -c example_config.yaml
#####################################  ROSie  #####################################
#  Version: 0.3.0                                                                 #
#  Date:    08.07.2025                                                            #
#  Time:    15:50:24                                                              #
#  Copyright (c) Siemens AG, 2025, and licensors. All rights reserved.            #
#  Portions include Open Source Software. See ReadMe_OSS_0.3.0.html for details.  #
###################################################################################

Environment variable path  does not exist.

```

#### ✔️ Possible Solution

As a solution, the install script `scripts/install_rosie_environment.sh` can be executed, or the environment variable `USER_ROSIE_WORKSPACE_PATH` (.env file) can be set manually.

### USER_ROS2_INSTALLATION_PATH

If you are using standard ROS 2 interface configurations e.g. [geometry_msgs/msg/Twist](https://github.com/ros2/common_interfaces/blob/humble/geometry_msgs/msg/Twist.msg), the `USER_ROS2_INSTALLATION_PATH` environment variable is crucial for ROSie to locate your standard ROS 2 installation directory.\
This variable should point to the root directory of the ROS 2 installation, where all `.msg` and `include/lib` definitions are located.

#### ⚠️ Variable Not Defined

If the `USER_ROS2_INSTALLATION_PATH` environment variable is not defined, ROSie will not be able to find necessary standard ROS 2 message definitions, leading to errors during gathering all data type information.\
The following error message will be displayed:

```bash
$ rosie -c example_config.yaml
#####################################  ROSie  #####################################
#  Version: 0.3.0                                                                 #
#  Date:    08.07.2025                                                            #
#  Time:    15:51:23                                                              #
#  Copyright (c) Siemens AG, 2025, and licensors. All rights reserved.            #
#  Portions include Open Source Software. See ReadMe_OSS_0.3.0.html for details.  #
###################################################################################

Gathering ROS 2 message types information ...
Environment variable  is not declared.

```

#### ⚠️ Wrong ROS 2 Installation Path

If the provided `USER_ROS2_INSTALLATION_PATH` points to a directory, which does not exist, the output should look like the following:

```bash
$ rosie -c example_config.yaml
#####################################  ROSie  #####################################
#  Version: 0.3.0                                                                 #
#  Date:    08.07.2025                                                            #
#  Time:    15:52:45                                                              #
#  Copyright (c) Siemens AG, 2025, and licensors. All rights reserved.            #
#  Portions include Open Source Software. See ReadMe_OSS_0.3.0.html for details.  #
###################################################################################

Gathering ROS 2 message types information ...
Environment variable path  does not exist.

```

#### ✔️ Possible Solution

You can solve those issues by either running the install script `scripts/install_rosie_environment.sh` or manually setting the environment variable `USER_ROS2_INSTALLATION_PATH`.\
With the manual step, the `USER_ROS2_INSTALLATION_PATH` variable needs to point to the correct path of your ROS 2 installation. This can be a standard ROS 2 installation like `/opt/ros/humble` or a source installation such as `/home/user/humble_ws/install`.\
Note that the naming extension `.../install` is important if you are using a custom ROS 2 installation.

### USER_ROS2_INTERFACE_INSTALLATION_PATH

In addition to standard ROS 2 message types, you can define and provide custom .msg definitions to ROSie.\
Therefore, the environment variable `USER_ROS2_INTERFACE_INSTALLATION_PATH` has to be set to provide the path to your custom ROS 2 interface installation directory e.g. `/home/user/ros2_ws`.

#### ⚠️ Type Not Found

If you defined custom ROS 2 .msg definitions and the `USER_ROS2_INTERFACE_INSTALLATION_PATH` environment variable is not defined, ROSie will not be able to find the corresponding custom ROS 2 message definitions, leading to errors during the gathering of all data type information.\
An output like the following will be displayed:

```bash
$ rosie -c custom_config.yaml
#####################################  ROSie  #####################################
#  Version: 0.3.0                                                                 #
#  Date:    08.07.2025                                                            #
#  Time:    15:53:36                                                              #
#  Copyright (c) Siemens AG, 2025, and licensors. All rights reserved.            #
#  Portions include Open Source Software. See ReadMe_OSS_0.3.0.html for details.  #
###################################################################################

Gathering ROS 2 message types information ...
Could not find ROS 2 type file . Make sure the ENV variable  is set if you provided a custom ROS 2 interface in the YAML config.

```

#### ✔️ Possible Solution

You can solve those issues by either running the install script `scripts/install_rosie_environment.sh` or manually setting the environment variable `USER_ROS2_INTERFACE_INSTALLATION_PATH`.

## Usage Errors

As ROSie is a command line tool, errors can occur easily and therefore, the following common errors can occur.

### Ownership of `start-rosie` Script

It could occur that the `start-rosie` script in `/usr/local/bin` has a problem with the ownership when using the binary installer.

```bash
$ start-rosie
/bin/bash: /usr/local/bin/start-rosie: Permission denied

```

#### ✔️ Possible Solution

You can fix it changing the ownership of the script:

```bash
sudo chown $USER:$USER /usr/local/bin/start-rosie

```

### No Configuration File

⚠️ ROSie is used with the `--config [-c]` parameter but without the required YAML configuration file name.

```bash
$ rosie -c
#####################################  ROSie  #####################################
#  Version: 0.3.0                                                                 #
#  Date:    08.07.2025                                                            #
#  Time:    15:54:01                                                              #
#  Copyright (c) Siemens AG, 2025, and licensors. All rights reserved.            #
#  Portions include Open Source Software. See ReadMe_OSS_0.3.0.html for details.  #
###################################################################################

Invalid option provided: missing argument for -c

```

✔️ Provide the exact name of the YAML configuration file that your want to use and which is placed in the `input directory` of your ROSie workspace.

### Wrong Configuration File Name

⚠️ ROSie is used with the `--config [-c]` parameter and a non-existing YAML configuration file.

```bash
$ rosie -c non_existing_config.yaml
#####################################  ROSie  #####################################
#  Version: 0.3.0                                                                 #
#  Date:    08.07.2025                                                            #
#  Time:    15:54:51                                                              #
#  Copyright (c) Siemens AG, 2025, and licensors. All rights reserved.            #
#  Portions include Open Source Software. See ReadMe_OSS_0.3.0.html for details.  #
###################################################################################

YAML configuration file  does not exist in workspace input directory.

```

✔️ Provide the name of a YAML configuration file name, which file exists in the ROSie workspace `input directory`.

### No PSC File

⚠️ ROSie is used with a valid YAML configuration file as well as the `--psc_file [-t]` parameter, but without a corresponding .psc file name.

```bash
$ rosie -c example_config.yaml -t
#####################################  ROSie  #####################################
#  Version: 0.3.0                                                                 #
#  Date:    08.07.2025                                                            #
#  Time:    15:55:27                                                              #
#  Copyright (c) Siemens AG, 2025, and licensors. All rights reserved.            #
#  Portions include Open Source Software. See ReadMe_OSS_0.3.0.html for details.  #
###################################################################################

Invalid option provided: missing argument for -t

```

✔️ Provide a .psc file, that exists in the ROSie workspace `input directory`.\
Note that this parameter combination only makes sense with the use of the [--non_interactive [-y]](usage/additional_resources/command_line_tool.html#disable-user-input-wizard) flag to go through the whole generation process without user interaction.

### Wrong PSC File Name

⚠️ ROSie is used with a valid non-interactive parameter combination but with providing a .psc file name, that either does not exist in the input directory or is spelled wrong.

```bash
$ rosie -y -c example_config.yaml -t non_existing_project.psc
#####################################  ROSie  #####################################
#  Version: 0.3.0                                                                 #
#  Date:    08.07.2025                                                            #
#  Time:    15:57:11                                                              #
#  Copyright (c) Siemens AG, 2025, and licensors. All rights reserved.            #
#  Portions include Open Source Software. See ReadMe_OSS_0.3.0.html for details.  #
###################################################################################

TIA project .psc file  does not exist in workspace input directory.

```

✔️ Provide a .psc file, that exists in the ROSie workspace `input directory` or is spelled correct.

### Missing PSC File in Non-Interactive Mode

⚠️ ROSie is used in non-interactive mode ([-y](usage/additional_resources/command_line_tool.html#disable-user-input-wizard)), which requires the .psc file an additional argument.

```bash
$ rosie -c example_config.yaml -y
#####################################  ROSie  #####################################
#  Version: 0.3.0                                                                 #
#  Date:    08.07.2025                                                            #
#  Time:    15:58:54                                                              #
#  Copyright (c) Siemens AG, 2025, and licensors. All rights reserved.            #
#  Portions include Open Source Software. See ReadMe_OSS_0.3.0.html for details.  #
###################################################################################

The argument --psc_file [t] and .psc file name must be set in non-interactive mode.

```

✔️ Extend the command with the parameter `--psc_file` or `-t` plus the name of the .psc file you want to use and which is placed in the ROSie workspace `input directory`.

### Wrong PSC File For Provided YAML Configuration

⚠️ ROSie is used in non-interactive mode with a YAML configuration, that does not match the provided .psc file. Therefore the ROSie API code fails to compile.

```bash
$ rosie -y -c project_1.yaml -t project_2.psc
#####################################  ROSie  #####################################
#  Version: 0.3.0                                                                 #
#  Date:    08.07.2025                                                            #
#  Time:    15:59:28                                                              #
#  Copyright (c) Siemens AG, 2025, and licensors. All rights reserved.            #
#  Portions include Open Source Software. See ReadMe_OSS_0.3.0.html for details.  #
###################################################################################

Gathering ROS 2 message types information ...
Generating PLC code ...
Generating ROSie API library and ROS 2 package ...
Could not generate ROSie API code.

```

✔️ Provide a .psc file, that was previously generated with the same YAML configuration based PLC code.

### ROSie API Code Generation Error

⚠️ During the ROS 2 package generation step, the ROSie API code generation fails.

```bash
$ rosie -y -c project_1.yaml -t project_1.psc
#####################################  ROSie  #####################################
#  Version: 0.3.0                                                                 #
#  Date:    08.07.2025                                                            #
#  Time:    16:04:32                                                              #
#  Copyright (c) Siemens AG, 2025, and licensors. All rights reserved.            #
#  Portions include Open Source Software. See ReadMe_OSS_0.3.0.html for details.  #
###################################################################################

Gathering ROS 2 message types information ...
Generating PLC code ...
Generating ROSie API library and ROS 2 package ...
Could not generate ROSie API code.

```

✔️ Take a look at the generated `/environment` directory, where a file named `_extended_rosie_logs.log` provides more detailed information about the actual error.\
Please contact us or your Siemens region in case the error can not be solved directly.

## FAQ

### How can I check which RIB version is installed on my IPC?

✔️ The actual RIB version can be checked via the `RIB_App` executable:

```bash
RIB_App --version

```

### How can I check which Industrial OS is installed on my IPC?

✔️ The Industrial OS version information can be displayed with the following command:

```bash
cat /etc/os-release

```

# Video Tutorial for ROSie Setup and Usage

This video serves as a brief tutorial on how to install and use ROSie.\
Please refer to the [setup](setup/setup.html) and [usage](usage/usage.html) section for a more detail explanation of the step by step procedure.

# Engineering System Setup Guide

⚠️ **Disclaimer: Why is Windows required? Because TIA Portal supports only Windows!**

Please note that this guide is meant as an **example** for the complete setup process of the engineering system. Different step combinations or the use of other similar tools may also be suitable.

Take into account that the steps shown in this guide may be outdated, for an up-to-date installation process, please visit each of the tools websites.

## Official Documentation Links

- [Windows Subsystem for Linux Documentation](https://learn.microsoft.com/en-us/windows/wsl/)
- [Visual Studio Code Documentation](https://code.visualstudio.com/docs)
- [ROS 2 Humble Documentation](https://docs.ros.org/en/humble/)
- [Docker Documentation](https://docs.docker.com/)
- [TIA Portal Documentation](https://support.industry.siemens.com/cs/document/109827383/installation-tia-portal-v19-step-7-wincc-other-sw)

## 1. Install Windows Subsystem for Linux (WSL)

### 1. Open PowerShell as Administrator

```powershell
wsl --install

```

### 2. Verify Installation and Update

```powershell
wsl --list --online
wsl --install -d Ubuntu

```

### 3. Restart your computer

### 4. Complete Ubuntu Setup

- Create a username when prompted
- Create a password
- Confirm password

*Reference: [WSL Installation Guide](https://learn.microsoft.com/en-us/windows/wsl/install)*

## 2. Install Visual Studio Code

### 1. Download VS Code

- Go to [VS Code Download Page](https://code.visualstudio.com/)
- Click "Download for Windows"

### 2. Install VS Code

- Run the downloaded installer
- Accept the agreement
- Select destination location
- Click "Next" and "Install"

### 3. Install Remote Development Extension

- Open VS Code
- Press `Ctrl+Shift+X`
- Search for "Remote Development"
- Click "Install" on "[Remote Development Extension Pack](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.vscode-remote-extensionpack)"

## 4. Connect to WSL from VS Code

- Click on the blue button in the lower-left corner of VS Code
- From the command palette that opens at the top, select "Connect to WSL"
- Alternative: Use keyboard shortcut: `Ctrl+Shift+P` and type "Connect to WSL"

## 5. Open Terminal in WSL

- Go to Terminal → New Terminal in the top menu

*Reference: [VS Code Setup Guide](https://code.visualstudio.com/docs/setup/windows)*

## 3. Install ROS 2 in WSL

Write the following commands into the terminal window opened in VS Code:

### 1. Set up sources

```bash
sudo apt update && sudo apt install -y software-properties-common
sudo add-apt-repository universe

```

### 2. Add ROS 2 GPG key

```bash
sudo curl -sSL https://raw.githubusercontent.com/ros/rosdistro/master/ros.key -o /usr/share/keyrings/ros-archive-keyring.gpg

```

### 3. Add repository to sources list

```bash
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/ros-archive-keyring.gpg] http://packages.ros.org/ros2/ubuntu $(. /etc/os-release && echo $UBUNTU_CODENAME) main" | sudo tee /etc/apt/sources.list.d/ros2.list > /dev/null

```

### 4. Install ROS 2

```bash
sudo apt update
sudo apt install -y ros-humble-desktop

```

### 5. Source ROS 2 environment

```bash
echo "source /opt/ros/humble/setup.bash" >> ~/.bashrc
source ~/.bashrc

```

*Reference: [ROS 2 Installation Guide](https://docs.ros.org/en/humble/Installation/Ubuntu-Install-Debians.html)*

## 4. Install Docker in WSL

Write the following commands into the open terminal window in VS Code:

### 1. Remove old versions

```bash
sudo apt-get remove docker docker-engine docker.io containerd runc docker-compose docker-compose-v2

```

### 2. Set up Docker repository

```bash
sudo apt-get update
sudo apt-get install -y \
      ca-certificates \
      curl \
      gnupg \
      lsb-release

```

### 3. Add Docker's official GPG key

```bash
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg

```

### 4. Install Docker Engine

```bash
sudo apt-get update
sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin

```

### 5. Configure Docker permissions

```bash
sudo usermod -aG docker $USER
newgrp docker

```

### 6. Test Docker installation

```bash
docker run hello-world

```

*Reference: [Docker Installation Guide](https://docs.docker.com/engine/install/ubuntu/)*

## 5. Install TIA Portal V20

### 1. Download TIA Portal V20

- Visit [Siemens Industry Online Support TIA V20 page](https://support.industry.siemens.com/cs/document/109963850/simatic-step-7-incl-safety-s7-plcsim-and-wincc-v20-trial-download)
- Download TIA Portal V20 installation files

### 2. Prepare for Installation

- Close all running applications
- Disable antivirus temporarily
- Run as Administrator

### 3. Installation Steps

- Double-click the installation file
- Select language
- Accept license agreement
- Choose installation path
- Select components
- Click "Install"

### 4. Post-Installation

- Activate your license
- Install any available updates
- Restart computer

### 5. Download and Install Update 1

- Visit [Siemens Industry Online Support TIA V20 update page](https://support.industry.siemens.com/cs/document/109963851/tia-portal-v20-updates?dti=0&lc=en-AE)
- Download the update package for STEP 7 / WinCC V20 Update 1
- Close TIA Portal if it's running
- Run the update installer as Administrator
- Follow the installation wizard
- Wait for the installation to complete
- Restart computer after installation

*Reference: [TIA Portal Installation Guide](https://support.industry.siemens.com/cs/document/109963850/simatic-step-7-incl-safety-s7-plcsim-and-wincc-v20-trial-download)*

# ROSie Setup

In case you haven't already, please refer to the [Engineering System Setup](engineering_system.html) in order to prepare all the necessary tools to use ROSie.\
Once the Engineering System is configured, copy the installer in the directory where you want to locate the ROSie generator.

To execute the installer you need to change its rights.

```bash
sudo chmod +x 

```

Run the installer with sudo privileges.

```bash
sudo bash 

```

The standard ROS 2 installation paths, `/opt/ros/humble` or `/opt/ros/jazzy`, will be detected automatically. Otherwise you will be asked to provide the path to your ROS 2 installation.

In case that multiple ROS 2 versions are installed in the standard paths, you will be asked for which of them you want to use before continuing.

```bash
Creating directory rosie_workspace
Verifying archive integrity...  100%   MD5 checksums are OK. All good.
Uncompressing Installer for ROSie version 0-3-0  100%  
Loaded image: cr.siemens.com/ros/projects/ros_oie/ros_plc_connector/deploy:0.3.0
Loaded image: cr.siemens.com/ros/projects/ros_oie/ros_plc_connector/deploy:0-3-0
Found ROS 2 installation at: /opt/ros/humble
Are you using custom ROS 2 message packages? (y/n):

```

Afterwards you will be asked if you are using any custom ROS 2 messages that are not part of the standard ROS 2 installation. If that is not the case answer with n and press enter.

```bash
Are you using custom ROS 2 message packages? (y/n): n

```

If you are using custom messages, answer with y and type in the path to the workspace folder of your compiled custom message package(s). Please note that the custom messages packages must already be built and the install folder must exist within this workspace path.

```bash
Are you using custom ROS 2 message packages? (y/n): y
Please provide the path to the workspace folder of the ROS 2 workspace that contains the built custom messages.
Path to custom message workspace folder (e.g. ros2_ws): /home/user/ros2_ws
Found custom message packages at: /home/user/ros2_ws

```

The ROSie workspace path will be set automatically to the directory of the 'package' folder.

If the setup was successful, you will see the following message:

```bash
Successfully installed ROSie environment.

```

If you want to move the ROSie generator to another location in the future, please rerun the install binary in the new location.\
After the installation process, the newly created `rosie_workspace` directory can be opened e.g. via [Visual Studio Code](https://code.visualstudio.com/).

The created `rosie_workspace` directory contains the following workspace structure:

Details

```bash
rosie_workspace  # The main workspace directory containing all subfolders of ROSie.
    |-- docker_images  # Contains the compressed docker image of the ROSie generator.
    |-- documentation  # Link to the online documentation of ROSie
    |-- input  # Input directory where the YAML configuration file and the .psc files have to be located.
        |-- valid_config_example.yaml  # Example configuration file.
    |-- oss  # Contains the used open source libraries and a OSS license overview file.
    |-- output  # Directory where every generated PLC code and ROS 2 package will be generated by ROSie under the directory name "date + timestamp".
    |-- scripts  # Contains the installation script to set up the ROSie environment on the user's system and to start ROSie.
    |-- template  # Directory with example TIA project files which can be used as templates for the user's own TIA project.
    |-- .env  # Environment configuration file for the ROSie generator. This file will be automatically edited by the installation script and stores the following environment variables:
              # [required] USER_ROS2_INSTALLATION_PATH: Path to the the users ROS 2 installation e.g. /opt/ros/humble.
              # [optional] USER_ROS2_INTERFACE_INSTALLATION_PATH: Path to the users custom interface installation path if custom .msg types are specified within the YAML config.
              # [required] USER_ROSIE_WORKSPACE_PATH: The path to this directory where the `.env` and `docker-compose.yaml` files are located.
    |-- docker-compose.yaml  # Docker Compose setup with predefined user volume mounting -> does not need to be changed.
    |-- README.md  # Entrypoint README for workspace descriptions

```

# Usage

## Prerequisites

Completed ROSie installer and workspace [setup](../setup/setup.html#rosie-setup).

## Generating ROS 2 - PLC Connection

### 1. Create interface description file config.yaml

At first a YAML file has to be created, which specifies the interfaces between ROS 2 and the used PLC program according to the rules described in the [YAML Configuration Structure](additional_resources/yaml_config_structure.html). This YAML file should be saved into the input folder in your ROSie Workspace.

A template for a working configuration looks like this:

```yaml
# System description of the generated PLC code and ROS 2 package
system:
  ros2:
    package_name: rosie_example_package # Generated ROS 2 package name
    node_name: rosie_node # Generated ROS 2 node name
  rib:
    RIB_App_IPv4: 127.0.0.1 # IPv4 address of the SHM control app: RIB_App
    RIB_App_port: 27567 # Port of the SHM control app: RIB_App
    RIB_Version: "v2.2.4" # Installed RIB version [v2.1.1, v2.2.4]

# All topics that subscribe to a ROS 2 topic and write its data to the PLC
ros2_to_plc:
  topics:
    - type: "geometry_msgs/msg/Twist.msg" # The full ROS 2 message type defintion
      ros2_topic: "/cmd_vel" # The ROS 2 topic name, also acts as DataBlock identifier in the PLC
      rate: 200.0 # in [Hz]

# All topics that read data from the PLC and publish it on a ROS 2 topic
plc_to_ros2:
  topics:
    - type: "sensor_msgs/msg/BatteryState.msg" # The full ROS 2 message type defintion
      ros2_topic: "/battery_state" # The ROS 2 topic name, also acts as DataBlock identifier in the PLC
      rate: 10.0 # in [Hz]

```

Please note that there is also a provided example file under

```bash
workspace/input/valid_config_example.yaml

```

### 2. Start ROSie container

If you completed the [ROSie setup](../setup/setup.html) successfully, you can now start the ROSie container by running the following command in the terminal:

```bash
start-rosie

```

Optional: Manual setup of the ROSie environment and start of the container In case the install script was not executed, the ROSie environment can also be set up manually. Navigate to the installed ROSie package:

```bash
cd path/to/rosie_workspace

```

Make sure the following Environment variables in the `.env` file inside the folder are set according to your ROS 2 environment.

```bash
# [REQUIRED]
# The users ROS 2 base installation path share directory that contains all standard interfaces including their .msg files.
# This directory has to be be mounted in the ROSie container to be able to lookup all standard types.
# Standard ROS 2 installation example: /opt/ros/jazzy
# Custom ROS 2 installation example:   /home/user/jazzy_ws
USER_ROS2_INSTALLATION_PATH=/opt/ros/jazzy

# [OPTIONAL]
# The user directory where the optionally needed custom ROS 2 interfaces are installed
# Example: /home/user/ros2_ws
# ros2_ws
#     |-- build
#     |-- install
#     |-- log
#     |-- src
#         |-- custom_msgs
USER_ROS2_INTERFACE_INSTALLATION_PATH=/home/user/ros2_ws

# [REQUIRED]
# The current directory where this .env file and the whole provided ROSie workspace is located
USER_ROSIE_WORKSPACE_PATH=/home/user/rosie_workspace

```

Afterwards start ROSie with:

```bash
docker compose run --name ROSie --rm ROSie

```

### 3. Generate PLC code using the ROSie command line tool

In the container environment, start the generation of the PLC code by choosing a configuration .yaml file located within the Workspace input folder.

```bash
rosie -c .yaml

```

If your configuration is valid the output should look similar to this:

```bash
$ rosie -c config.yaml
#####################################  ROSie  #####################################
#  Version: 0.3.0                                                                 #
#  Date:    08.07.2025                                                            #
#  Time:    14:21:38                                                              #
#  Copyright (c) Siemens AG, 2025, and licensors. All rights reserved.            #
#  Portions include Open Source Software. See ReadMe_OSS_0.3.0.html for details.  #
###################################################################################

Gathering ROS 2 message types information ...
Generating PLC code ...
Please import the generated PLC code into your TIA project, export the TIA project .psc file, and place into the ROSie input folder. 
Please provide the file name of the .psc in the prompt when requested, e.g., 'tia_project.psc' and press ENTER to continue the generation process or 'n' and ENTER to exit the program.

```

Inside the output folder of your ROSie workspace a time-stamped folder has been created, that contains the generated PLC code. The path to the generated PLC sources is similar to the following:

```bash
workspace/output/time-stamp/output/plc_code/_rosie_generated_rib_scl_structs.scl

```

### 4. Move generated PLC code to TIA V20 engineering machine

Now move the generated PLC source file `_rosie_generated_rib_scl_structs.scl` to your TIA engineering machine. If you use a virtual machine you can use shared folders for example.

### 5. Import generated PLC source as external source file

Open the TIA project of your IPC or OpenController that shall establish a connection to ROS 2. Select the Software-Controller of the device and import the previously generated PLC source file into your TIA V20 project.

### 6. Generate blocks from external PLC source file

The imported source file can now be used to generate the PLC program blocks of the ROS 2 \<-> PLC connection:

There will be a warning message informing you that any duplicate blocks will be overwritten. Make sure to have a backup of your project and acknowledge the warning dialog.

Afterwards the following blocks are generated into your TIA project:

And the following UDTs representing the ROS 2 msg types inside your TIA portal:

### 7. Create .psc file of the device containing the ROSie generated blocks

To proceed with the generation of the ROS 2 package of the ROS 2 \<-> PLC connection a .psc file has to be created from your device inside the TIA project.

This can be done by pressing on Project/Memory card file/New/PC system configuration file:

Select the name of the .psc file. Don't use whitespaces inside the filename as this file will be used in a Linux-based OS in another step of the generation process.

Load your OpenController or IPC device into the .psc file by dragging and dropping the device onto the created Memory Card file:

A dialog will open. Press on load to load the device configuration into the .psc file.

### 8. Move .psc file to ROSie input directory

Now copy the created .psc file into the input directory of your ROSie workspace. If you are using a virtual machine you can use shared folders, or if using WSL, you can directly save into your WSL filesystem (normally found under \\wsl$\\Ubuntu)

### 9. Use .psc file to continue ROSie process to generate ROS 2 package

As the .psc file is now located in the ROSie context, you can now proceed with the generation of the ros2 package. The ROSie cli tool should still be waiting for the .psc file, at the state of the end of [Step3](#3-generate-plc-code-using-the-rosie-command-line-tool). Type in the name of the .psc file you created and press enter.

```bash
$ rosie -c config.yaml
#####################################  ROSie  #####################################
#  Version: 0.3.0                                                                 #
#  Date:    08.07.2025                                                            #
#  Time:    14:26:42                                                              #
#  Copyright (c) Siemens AG, 2025, and licensors. All rights reserved.            #
#  Portions include Open Source Software. See ReadMe_OSS_0.3.0.html for details.  #
###################################################################################

Gathering ROS 2 message types information ...
Generating PLC code ...
Please import the generated PLC code into your TIA project, export the TIA project .psc file, and place into the ROSie input folder. 
Please provide the file name of the .psc in the prompt when requested, e.g., 'tia_project.psc' and press ENTER to continue the generation process or 'n' and ENTER to exit the program.
plc-system-config.psc

```

If the generation was successful, the output of the cli tool should look similar to this:

```bash
$ rosie -c config.yaml
#####################################  ROSie  #####################################
#  Version: 0.3.0                                                                 #
#  Date:    08.07.2025                                                            #
#  Time:    14:28:35                                                              #
#  Copyright (c) Siemens AG, 2025, and licensors. All rights reserved.            #
#  Portions include Open Source Software. See ReadMe_OSS_0.3.0.html for details.  #
###################################################################################

Gathering ROS 2 message types information ...
Generating PLC code ...
Please import the generated PLC code into your TIA project, export the TIA project .psc file, and place into the ROSie input folder. 
Please provide the file name of the .psc in the prompt when requested, e.g., 'tia_project.psc' and press ENTER to continue the generation process or 'n' and ENTER to exit the program.
plc-system-config.psc
User Input: plc-system-config.psc
TIA project .psc file  found in input workspace. Continuing generation process ...
Generating ROSie API library and ROS 2 package ...
Successfully generated ROS 2 package and PLC code.

```

### 10. Move generated ROS 2 package to your desired workspace and build the package

Congratulations the generation process is now done. You can find the generated ROS 2 package in the output folder of your ROSie workspace under:

```bash
output//output/ros2_deployment/ros2_ws/src/ros2_package_name

```

Additionally, in the same output folder you will find the configuration .yaml file that was used for the creation of the ROS 2 package, as well as logs from the creation process in case of errors.

Please be sure to exit the ROSie container before continuing the process of moving your generated package. Otherwise some problems might arise while moving files around. To exit the container simply type `exit` into the command line.

#### a. Creating a new container based ROS 2 Workspace

Within the `output/ros2_deployment` folder, a Dockerfile, docker compose file and build script were also automatically generated. These files can be used to build a docker image as archive which can then be loaded into your runtime system.

Navigate to the ros2_deployment folder and run the following command to build the generated ROS 2 package as a deployable image:

```bash
# build_deploy_container.sh parameters:
# ros2-version: ROS 2 version [supported: humble, jazzy, kilted], default: jazzy
# container:    Container name, default: rosie_bridge_example
bash build_deploy_container.sh --ros2-version jazzy --container rosie_bridge_example

```

The script will build a docker image based on the standard ROS 2 image you have selected e.g. Jazzy, then compile the ROS 2 packages and setting up the required RIB runtime dependencies for the generated ROSie node. Afterwards, the `docker-compose.yaml` file will be updated to use the generated ROS 2 package.\
Finally, the script will export the generated docker image as a .tar archive with the name `_.tar`.

You can then follow these steps to deploy the generated image archive into your runtime system:

Copy the docker image archive to the Industrial OS.

```bash
scp rosie_bridge_example_jazzy.tar [USER]@[DESTINATION_HOST_IP]:rosie_bridge_example_jazzy.tar

```

Copy the docker-compose.yaml file to the Industrial OS.

```bash
scp docker-compose.yaml [USER]@[DESTINATION_HOST_IP]:docker-compose.yaml

```

Load the image again in the Industrial OS.

```bash
ssh [USER]@[DESTINATION_HOST_IP]
docker load < rosie_bridge_example_jazzy.tar 

```

Now you are ready to run the generated ROS 2 package on your IPC via the generated `docker-compose.yaml` and docker compose.

#### b. Working on an existing ROS 2 Workspace

Although using the deploy process of docker images is recommended, you can use your existing ROS 2 workspace within Industrial OS. Therefore, all that is needed is to copy the complete ROS 2 package directory to your IPC runtime system's ROS 2 workspace and compile it via `colcon build`.

## Running the Generated Interface

### 1. Start RIB_App

Start the `RIB_App` executable inside the Industrial OS of your IPC device.

```bash
RIB_App

```

This application serves as a RIB participant broker like shown in the following illustration:

### 2. Load PLC program to the Software PLC

If not done already, load the TIA project that was extended by the ROSie generated program blocks to the Software PLC.\
The generated state machine will then try to automatically connect to the `RIB_App`.

### 3. Start ROS 2 Node

Start the ROS 2 node using the generated `docker-compose.yaml` file in your Industrial OS:

```bash
ssh [USER]@[DESTINATION_HOST_IP]
docker compose up

```

# Command Line Interface

## Required Argument

### YAML Configuration File

The only required parameter is the `--config` or the short version `-c` to provide a predefined YAML configuration file to ROSie for code generation.\
To run ROSie with a predefined YAML and start the whole generation process, run the following command:

```bash
rosie --config file_name.yaml

```

Example console output:

```bash
$ rosie -c example_config.yaml
#####################################  ROSie  #####################################
#  Version: 0.3.0                                                                 #
#  Date:    08.07.2025                                                            #
#  Time:    15:53:21                                                              #
#  Copyright (c) Siemens AG, 2025, and licensors. All rights reserved.            #
#  Portions include Open Source Software. See ReadMe_OSS_0.3.0.html for details.  #
###################################################################################

Gathering ROS 2 message types information ...
Generating PLC code ...
Please import the generated PLC code into your TIA project, export the TIA project .psc file, and place into the ROSie input folder. 
Please provide the file name of the .psc in the prompt when requested, e.g., 'tia_project.psc' and press ENTER to continue the generation process or 'n' and ENTER to exit the program.

```

## Optional Arguments

### Verify YAML Config File

The `--verify` parameter, short version `-v`, can be set to check, if the provided YAML config meets the requirements.\
To verify, if the provided YAML config matches the required syntax, run the following command:

```bash
rosie --verify --config file_name.yaml

```

Example console output:

```bash
$ rosie -v -c example_config.yaml
#####################################  ROSie  #####################################
#  Version: 0.3.0                                                                 #
#  Date:    08.07.2025                                                            #
#  Time:    15:54:30                                                              #
#  Copyright (c) Siemens AG, 2025, and licensors. All rights reserved.            #
#  Portions include Open Source Software. See ReadMe_OSS_0.3.0.html for details.  #
###################################################################################

YAML verifying and parsing successful.

```

### Disable User Input Wizard

The `--non_interactive` parameter, short version `-y`, can be used to skip all user interactions and process the whole generation process automatically.\
To start the whole generation process non-interactively, additionally the [--psc_file](#psc-file) argument needs to be provided:

```bash
rosie --non_interactive --config file_name.yaml --psc_file tia_project.psc

```

Example console output:

```bash
$ rosie -y -c example_config.yaml -t example_tia_project.psc
#####################################  ROSie  #####################################
#  Version: 0.3.0                                                                 #
#  Date:    08.07.2025                                                            #
#  Time:    15:59:17                                                              #
#  Copyright (c) Siemens AG, 2025, and licensors. All rights reserved.            #
#  Portions include Open Source Software. See ReadMe_OSS_0.3.0.html for details.  #
###################################################################################

Gathering ROS 2 message types information ...
Generating PLC code ...
Generating ROSie API library and ROS 2 package ...
Successfully generated ROS 2 package and PLC code.

```

### PSC File

The `--psc_file` parameter, short version `-t`, can be used to process all code generation steps without user interaction.\
It will provide the TIA Portal generated PC System Configuration (.psc) file for internal struct comparison against the PLC based data type definitions.\
⚠️ This argument needs to be combined with the [--non_interactive](#disable-user-input-wizard) argument.

Example usage (creates the same command line output as in the section [Disable User Input Wizard](#disable-user-input-wizard)):

```bash
rosie --non_interactive --config file_name.yaml --psc_file tia_project.psc

```

### Usage Help

Calling ROSie without any arguments or with the `--help` argument, short version `-h`, will print command line usage information:

```bash
rosie --help

```

Example console output:

```bash
$ rosie -h
#####################################  ROSie  #####################################
#  Version: 0.3.0                                                                 #
#  Date:    08.07.2025                                                            #
#  Time:    16:05:02                                                              #
#  Copyright (c) Siemens AG, 2025, and licensors. All rights reserved.            #
#  Portions include Open Source Software. See ReadMe_OSS_0.3.0.html for details.  #
###################################################################################

Command line parameters: 
  --config [-c]           YAML configuration file name, e.g., 'config.yaml' which defines system specifications (required)
  --help [-h]             Get information about the ROSie usage and CLI parameters
  --non_interactive [-y]  Disable the wizard mode and run all stages without user interaction (optional)
  --psc_file [-t]         TIA project .psc file name, e.g., 'tia_project.psc' (optional)
  --verify [-v]           Verification of the provided YAML configuration file (optional)

```

# YAML Configuration Structure

## Introduction

As ROSie provides a command line based interface for users, a service was needed to give users the possibility to specify their system using a [YAML](https://yaml.org) based configuration file.\
In this configuration, users can declare the [system](#system), which stands for the ROS 2 package that will be the wrapper for all defined data structures. Additionally, users shall define the [data structure connections](#topic) based on ROS 2 topics.

## System

An example of a system definition is shown below.\
The generated package will setup a fully functional ROS 2 package, that works out out of the box by using the parameters specified below.

```yaml
system:
  ros2:
    package_name: example_package
    node_name: example_node
  rib:
    RIB_App_IPv4: 127.0.0.1
    RIB_App_port: 27567
    RIB_Version: "v2.2.4"
  # optional
  plc:
    # optional
    rib_config_db_name: "rib_config_topic_DB"
    rib_connect_state_machine_fc_name: "rib_connect_build_up_FC"

```

### `system (required)`

The `system` keyword specifies the content of the generated ROS 2 package.

```yaml
system:

```

### `ros2 (required)`

The `ros2` keyword declares all content that is ROS 2 specific.

```yaml
  ros2:

```

### `package_name (required)`

The `package_name` keyword is used to define the name for the generated ROS 2 package.

```yaml
    package_name: 

```

### `node_name (required)`

The `node_name` keyword is used to define the name for the generated ROS 2 node inside the package.

```yaml
    node_name: 

```

### `rib (required)`

The `rib` keyword defines the connection to the RIB_App, that is used to manage different shared memory connections of the RIB.

```yaml
  rib:

```

### `RIB_App_IPv4 (required)`

The `RIB_App_IPv4` keyword specifies the IPv4 address of the RIB_App, to which the RIB API will be connected to via TCP based connection.\
Valid values are either `localhost` or a correctly set IPv4 address e.g. `127.0.0.1`.\
Note: The value can later be changed in the source code.

```yaml
    RIB_App_IPv4: localhost

```

### `RIB_App_port (required)`

The `RIB_App_port` keyword is used to define the TCP based port, at which the RIB_App is reachable.\
Valid are full numbers e.g. `80`.\
Note: The value can later be changed in the source code.

```yaml
    RIB_App_port: 

```

### `RIB_Version (required)`

The `RIB_Version` keyword specifies the RIB version which is being used by the system, e.g. `v2.2.4`.\
In order to get the RIB version currently installed in your IPC, the [command described in the FAQ section](../../troubleshooting.html#how-can-i-check-which-rib-version-is-installed-on-my-ipc) can be used.

```yaml
    RIB_Version: 

```

### `plc (optional)`

The `plc` keyword declares all content that is PLC specific.

```yaml
  plc:

```

### `rib_config_db_name (optional)`

The `rib_config_db_name` keyword can be used to change the name of the data block associated to the connection control and status of the RIB. The default value is "RIB_CONFIG_DB".

```yaml
    rib_config_db_name: 

```

### `rib_connect_state_machine_fc_name (optional)`

The `rib_connect_state_machine_fc_name` keyword can be used to change the name of the function block handling the automatic connection build up to the RIB. The default value is "RIB_CONNECT_STATE_MACHINE_FC".

```yaml
    rib_connect_state_machine_fc_name: 

```

## Topic

The `Topic` area relies on the principles of the [ROS 2 topic system](https://docs.ros.org/en/humble/Tutorials/Beginner-CLI-Tools/Understanding-ROS2-Topics/Understanding-ROS2-Topics.html) and is divided into two possible connection directions, either from [ROS 2 to PLC](#ros-2-to-plc) or from [PLC to ROS 2](#plc-to-ros-2).\
Those connections are based on the data structures that are defined within them.

### ROS 2 to PLC

To define a ROS 2 subscriber that writes its received data to the PLC, the `ros2_to_plc` tag shall be used.\
Within it, all needed data structures can be specified. Each type specification will be used to create a corresponding subscriber within the generated ROS 2 package.\
An example is provided below.

```yaml
ros2_to_plc:
  topics:
    - type: "custom_msgs/msg/Type.msg"
      ros2_topic: "/topic"
      rate: 100.0
    - type: "custom_msgs/msg/AnotherType.msg"
      ros2_topic: "/another_topic"
      rate: 5.0

```

#### `ros2_to_plc (required)`

The `ros2_to_plc` keyword defines the connection direction from ROS 2 to the PLC.

```yaml
ros2_to_plc:

```

#### `topics (required)`

The `topics` keyword is used to clarify, that the created ROS 2 part will be a subscriber, that listens on a specified topic.

```yaml
  topics:

```

#### `type (required)`

The `type` keyword defines the ROS 2 based topic message type, that will be searched for either in the provided custom interface path or the ROS 2 standard installation path.

```yaml
    - type: "/msg/.msg"

```

✔️ Valid values are:

- In case of a standard ROS 2 based message type e.g. geometry_msgs/msg/Twist located at the path specified via `USER_ROS2_INSTALLATION_PATH` ENV variable:\
  `"geometry_msgs/msg/Twist.msg"`
- In case of a custom message type from a custom interface package located at the path specified via `USER_ROS2_INTERFACE_INSTALLATION_PATH` ENV variable:\
  `"custom_msgs/msg/Type.msg"`

❌ Wrong values are e.g.:

- `"geometry_msgs/msg/Twist"`
- `"geometry_msgs/Twist"`

#### `ros2_topic (required)`

The `ros2_topic` keyword defines the ROS 2 topic name of the desired message type, where the '/' is optional.\
⚠️ The name must be unique in the YAML configuration!

```yaml
      ros2_topic: "/"

```

ROS 2 topic namespaces can be provided e.g.:

```yaml
      ros2_topic: "robot_1/topic_name"

```

#### `rate (required)`

The `rate` keyword is used to define the rate in Hz, in which messages will be written to the PLC on the specified topic.\
It is used to configure the interrupt cycle of the PLC side for this topic.

```yaml
      rate: 10.0 # in Hz

```

✔️ Valid subscription rates [Hz] are (as floating point values):

- All values > 0.0
- All values \<= 1000.0

⚠️ Values must not match the exact real subscription rate, but one has to keep in mind, that each generated PLC interrupt OB will provide a cycle time with the same rate as specified in the YAML!

### PLC to ROS 2

To define a ROS 2 publisher that reads the data received from the PLC and publishes it to ROS 2, the `plc_to_ros2` tag shall be used.\
Within it, all needed data structures can be specified similar to the configurations in the section [ROS 2 to PLC](#ros-2-to-plc). Each type specification will be used to create a corresponding publisher within the generated ROS 2 package.\
An example is provided below.

```yaml
plc_to_ros2:
  topics:
    - type: "custom_msgs/msg/Type.msg"
      ros2_topic: "/topic"
      rate: 250.0
    - type: "sensor_msgs/msg/BatteryState.msg"
      ros2_topic: "/battery"
      rate: 1.0

```

#### `plc_to_ros2 (required)`

The `plc_to_ros2` keyword defines the connection direction from the PLC to ROS 2.

```yaml
plc_to_ros2:

```

#### `topics (required)`

The `topics` keyword is used to clarify, that the created ROS 2 part will be a publisher, that publishes PLC data on a specified topic.

```yaml
  topics:

```

#### `type (required)`

The `type` keyword defines the ROS 2 based topic message type, that will be searched for either in the provided custom interface path or the ROS 2 standard installation path.

```yaml
    - type: "/msg/.msg"

```

✔️ Valid values are:

- In case of a standard ROS 2 based message type e.g. geometry_msgs/msg/Twist located at the path specified via `USER_ROS2_INSTALLATION_PATH` ENV variable:\
  `"geometry_msgs/msg/Twist.msg"`
- In case of a custom message type from a custom interface package located at the path specified via `USER_ROS2_INTERFACE_INSTALLATION_PATH` ENV variable:\
  `"custom_msgs/msg/Type.msg"`

❌ Wrong values are e.g.:

- `"geometry_msgs/msg/Twist"`
- `"geometry_msgs/Twist"`

#### `ros2_topic (required)`

The `ros2_topic` keyword defines the ROS 2 topic name of the desired message type, where the '/' is optional.\
⚠️ The name must be unique in the YAML configuration!

```yaml
      ros2_topic: "/"

```

ROS 2 topic namespaces can be provided e.g.:

```yaml
      ros2_topic: "robot_1/topic_name"

```

#### `rate (required)`

The `rate` keyword is used to define the rate in Hz, in which messages will be read from the PLC on the specified topic.\
It is used to configure the interrupt cycle of the PLC side as well as a ROS 2 based timer callback to read the data at the provided rate.

```yaml
      rate: 10.0 # in Hz

```

✔️ Valid subscription rates [Hz] are (as floating point values):

- All values > 0.0
- All values \<= 1000.0

⚠️ Values must not match the exact real subscription rate, but one has to keep in mind, that each generated PLC interrupt OB will provide a cycle time with the same rate as specified in the YAML!

## Example

Based on provided message type definitions in a custom `interface_package`, a ROSie YAML configuration `config.yaml` could look like the following:

```yaml
system:
  ros2:
    package_name: rosie_ros2_package
    node_name: rosie_ros2_node
  rib:
    RIB_App_IPv4: 127.0.0.1
    RIB_App_port: 27567
    RIB_Version: "v2.2.4"

ros2_to_plc:
  topics:
    - type: "interface_package/msg/Type.msg"
      ros2_topic: "/topic"
      rate: 50.0
    - type: "interface_package/msg/AnotherType.msg"
      ros2_topic: "/another_topic"
      rate: 10.0
    - type: "gemetry_msgs/msg/Twist.msg"
      ros2_topic: "cmd_vel"
      rate: 100.0

plc_to_ros2:
  topics:
    - type: "interface_package/msg/Type2.msg"
      ros2_topic: "namespaced/topic2"
      rate: 250.0
    - type: "geometry_msgs/msg/PoseStamped.msg"
      ros2_topic: "/pose"
      rate: 50.0

```

# Engineering System Setup Guide

⚠️ **Disclaimer: Why is Windows required? Because TIA Portal supports only Windows!**

Please note that this guide is meant as an **example** for the complete setup process of the engineering system. Different step combinations or the use of other similar tools may also be suitable.

Take into account that the steps shown in this guide may be outdated, for an up-to-date installation process, please visit each of the tools websites.

## Official Documentation Links

- [Windows Subsystem for Linux Documentation](https://learn.microsoft.com/en-us/windows/wsl/)
- [Visual Studio Code Documentation](https://code.visualstudio.com/docs)
- [ROS 2 Jazzy Documentation](https://docs.ros.org/en/jazzy/)
- [Docker Documentation](https://docs.docker.com/)
- [TIA Portal Documentation](https://support.industry.siemens.com/cs/document/109827383/installation-tia-portal-v19-step-7-wincc-other-sw)

## 1. Install Windows Subsystem for Linux (WSL)

### 1. Open PowerShell as Administrator

```powershell
wsl --install

```

### 2. Verify Installation and Update

```powershell
wsl --list --online
wsl --install -d Ubuntu

```

### 3. Restart your computer

### 4. Complete Ubuntu Setup

- Create a username when prompted
- Create a password
- Confirm password

*Reference: [WSL Installation Guide](https://learn.microsoft.com/en-us/windows/wsl/install)*

## 2. Install Visual Studio Code

### 1. Download VS Code

- Go to [VS Code Download Page](https://code.visualstudio.com/)
- Click "Download for Windows"

### 2. Install VS Code

- Run the downloaded installer
- Accept the agreement
- Select destination location
- Click "Next" and "Install"

### 3. Install Remote Development Extension

- Open VS Code
- Press `Ctrl+Shift+X`
- Search for "Remote Development"
- Click "Install" on "[Remote Development Extension Pack](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.vscode-remote-extensionpack)"

## 4. Connect to WSL from VS Code

- Click on the blue button in the lower-left corner of VS Code
- From the command palette that opens at the top, select "Connect to WSL"
- Alternative: Use keyboard shortcut: `Ctrl+Shift+P` and type "Connect to WSL"

## 5. Open Terminal in WSL

- Go to Terminal → New Terminal in the top menu

*Reference: [VS Code Setup Guide](https://code.visualstudio.com/docs/setup/windows)*

## 3. Install ROS 2 in WSL

Write the following commands into the terminal window opened in VS Code:

### 1. Set up sources

```bash
sudo apt update && sudo apt install -y software-properties-common
sudo add-apt-repository universe

```

### 2. Add ROS 2 GPG key

```bash
sudo curl -sSL https://raw.githubusercontent.com/ros/rosdistro/master/ros.key -o /usr/share/keyrings/ros-archive-keyring.gpg

```

### 3. Add repository to sources list

```bash
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/ros-archive-keyring.gpg] http://packages.ros.org/ros2/ubuntu $(. /etc/os-release && echo $UBUNTU_CODENAME) main" | sudo tee /etc/apt/sources.list.d/ros2.list > /dev/null

```

### 4. Install ROS 2

```bash
sudo apt update
sudo apt install -y ros-jazzy-desktop

```

### 5. Source ROS 2 environment

```bash
echo "source /opt/ros/jazzy/setup.bash" >> ~/.bashrc
source ~/.bashrc

```

*Reference: [ROS 2 Installation Guide](https://docs.ros.org/en/jazzy/Installation/Ubuntu-Install-Debians.html)*

## 4. Install Docker in WSL

Write the following commands into the open terminal window in VS Code:

### 1. Remove old versions

```bash
sudo apt-get remove docker docker-engine docker.io containerd runc docker-compose docker-compose-v2

```

### 2. Set up Docker repository

```bash
sudo apt-get update
sudo apt-get install -y \
      ca-certificates \
      curl \
      gnupg \
      lsb-release

```

### 3. Add Docker's official GPG key

```bash
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg

```

### 4. Install Docker Engine

```bash
sudo apt-get update
sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin

```

### 5. Configure Docker permissions

```bash
sudo usermod -aG docker $USER
newgrp docker

```

### 6. Test Docker installation

```bash
docker run hello-world

```

*Reference: [Docker Installation Guide](https://docs.docker.com/engine/install/ubuntu/)*

## 5. Install TIA Portal V20

### 1. Download TIA Portal V20

- Visit [Siemens Industry Online Support TIA V20 page](https://support.industry.siemens.com/cs/document/109963850/simatic-step-7-incl-safety-s7-plcsim-and-wincc-v20-trial-download)
- Download TIA Portal V20 installation files

### 2. Prepare for Installation

- Close all running applications
- Disable antivirus temporarily
- Run as Administrator

### 3. Installation Steps

- Double-click the installation file
- Select language
- Accept license agreement
- Choose installation path
- Select components
- Click "Install"

### 4. Post-Installation

- Activate your license
- Install any available updates
- Restart computer

### 5. Download and Install Latest TIA Portal Update

- Visit [Siemens Industry Online Support TIA V20 update page](https://support.industry.siemens.com/cs/document/109963851/tia-portal-v20-updates?dti=0&lc=en-AE)
- Download the update package for STEP 7 / WinCC V20 Update X
- Close TIA Portal if it's running
- Run the update installer as Administrator
- Follow the installation wizard
- Wait for the installation to complete
- Restart computer after installation

*Reference: [TIA Portal Installation Guide](https://support.industry.siemens.com/cs/document/109963850/simatic-step-7-incl-safety-s7-plcsim-and-wincc-v20-trial-download)*

# ROXSIE Setup

In case you haven't already, please refer to the [Engineering System Setup](engineering_system.html) in order to prepare all the necessary tools to use ROXSIE.\
Once the Engineering System is configured, copy the installer in the directory where you want to locate the ROXSIE generator.

To execute the installer you need to change its rights.

```bash
sudo chmod +x 

```

Run the installer with sudo privileges.

```bash
sudo bash 

```

Standard ROS 2 installation paths, e.g. `/opt/ros/jazzy`, will be detected automatically. Otherwise you will be asked to provide the path to your ROS 2 installation.

In case that multiple ROS 2 versions are installed in the standard paths, you will be asked for which of them you want to use before continuing.

```bash
Creating directory roxsie_workspace
Verifying archive integrity...  100%   MD5 checksums are OK. All good.
Uncompressing Installer for ROXSIE version 0-3-1  100%  
Loaded image: cr.siemens.com/ros/projects/ros_oie/ros_plc_connector/deploy:0.3.1
Loaded image: cr.siemens.com/ros/projects/ros_oie/ros_plc_connector/deploy:0-3-1
Found ROS 2 installation at: /opt/ros/jazzy
Are you using custom ROS 2 message packages? (y/n):

```

Afterwards you will be asked if you are using any custom ROS 2 messages that are not part of the standard ROS 2 installation. If that is not the case answer with n and press enter.

```bash
Are you using custom ROS 2 message packages? (y/n): n

```

If you are using custom messages, answer with y and type in the path to the workspace folder of your compiled custom message package(s). Please note that the custom messages packages must already be built and the install folder must exist within this workspace path.

```bash
Are you using custom ROS 2 message packages? (y/n): y
Please provide the path to the workspace folder of the ROS 2 workspace that contains the built custom messages.
Path to custom message workspace folder (e.g. ros2_ws): /home/user/ros2_ws
Found custom message packages at: /home/user/ros2_ws

```

The ROXSIE workspace path will be set automatically to the directory of the 'package' folder.

If the setup was successful, you will see the following message:

```bash
Successfully installed ROXSIE environment.

```

If you want to move the ROXSIE generator to another location in the future, please rerun the install binary in the new location.\
After the installation process, the newly created `roxsie_workspace` directory can be opened e.g. via [Visual Studio Code](https://code.visualstudio.com/).

The created `roxsie_workspace` directory contains the following workspace structure:

Details

```bash
roxsie_workspace  # The main workspace directory containing all subfolders of ROXSIE.
    |-- docker_images  # Contains the compressed docker image of the ROXSIE generator.
    |-- documentation  # Link to the online documentation of ROXSIE
    |-- input  # Input directory where the YAML configuration file and the .psc files have to be located.
        |-- valid_config_example.yaml  # Example configuration file.
    |-- oss  # Contains the used open source libraries and a OSS license overview file.
    |-- output  # Directory where every generated PLC code and ROS 2 package will be generated by ROXSIE under the directory name "date + timestamp".
    |-- scripts  # Contains the installation script to set up the ROXSIE environment on the user's system and to start ROXSIE.
    |-- template  # Directory with example TIA project files which can be used as templates for the user's own TIA project.
    |-- .env  # Environment configuration file for the ROXSIE generator. This file will be automatically edited by the installation script and stores the following environment variables:
              # [required] USER_ROS2_INSTALLATION_PATH: Path to the the users ROS 2 installation e.g. /opt/ros/jazzy.
              # [optional] USER_ROS2_INTERFACE_INSTALLATION_PATH: Path to the users custom ROS 2 workspace path if custom .msg types are specified within the YAML config.
              # [required] USER_ROXSIE_WORKSPACE_PATH: The path to this directory where the `.env` and `docker-compose.yaml` files are located.
    |-- docker-compose.yaml  # Docker Compose setup with predefined user volume mounting -> does not need to be changed.
    |-- README.md  # Entrypoint README for workspace descriptions

```

# Usage

## Prerequisites

Completed ROXSIE installer and workspace [setup](../setup/setup.html#roxsie-setup).

## Generating ROS 2 - PLC Connection

### 1. Create interface description file config.yaml

At first a YAML file has to be created, which specifies the interfaces between ROS 2 and the used PLC program according to the rules described in the [YAML Configuration Structure](additional_resources/yaml_config_structure.html). This YAML file should be saved into the input folder in your ROXSIE Workspace.

A template for a working configuration looks like this:

```yaml
# System description of the generated PLC code and ROS 2 package
system:
  ros2:
    package_name: roxsie_example_package # Generated ROS 2 package name
    node_name: roxsie_node # Generated ROS 2 node name
  rib:
    RIB_App_IPv4: 127.0.0.1 # IPv4 address of the SHM control app: RIB_App
    RIB_App_port: 27567 # Port of the SHM control app: RIB_App
    RIB_Version: "v2.2.4" # Installed RIB version [v2.1.1, v2.2.4]

# All topics that subscribe to a ROS 2 topic and write its data to the PLC
ros2_to_plc:
  topics:
    - type: "geometry_msgs/msg/Twist.msg" # The full ROS 2 message type defintion
      ros2_topic: "/cmd_vel" # The ROS 2 topic name, also acts as DataBlock identifier in the PLC
      rate: 200.0 # in [Hz]

# All topics that read data from the PLC and publish it on a ROS 2 topic
plc_to_ros2:
  topics:
    - type: "sensor_msgs/msg/BatteryState.msg" # The full ROS 2 message type defintion
      ros2_topic: "/battery_state" # The ROS 2 topic name, also acts as DataBlock identifier in the PLC
      rate: 10.0 # in [Hz]

```

Please note that there is also a provided example file under

```bash
workspace/input/valid_config_example.yaml

```

### 2. Start ROXSIE container

If you completed the [ROXSIE setup](../setup/setup.html) successfully, you can now start the ROXSIE container by running the following command in the terminal:

```bash
start-roxsie

```

Optional: Manual setup of the ROXSIE environment and start of the container In case the install script was not executed, the ROXSIE environment can also be set up manually. Navigate to the installed ROXSIE package:

```bash
cd path/to/roxsie_workspace

```

Make sure the following Environment variables in the `.env` file inside the folder are set according to your ROS 2 environment.

```bash
# [REQUIRED]
# The users ROS 2 base installation path share directory that contains all standard interfaces including their .msg files.
# This directory has to be be mounted in the ROXSIE container to be able to lookup all standard types.
# Standard ROS 2 installation example: /opt/ros/jazzy
# Custom ROS 2 installation example:   /home/${USER}/jazzy_ws
USER_ROS2_INSTALLATION_PATH=/opt/ros/jazzy

# [OPTIONAL]
# The user directory where the optionally needed custom ROS 2 interfaces are installed.
# Example: /home/${USER}/ros2_ws
# ros2_ws
#     |-- build
#     |-- install
#     |-- log
#     |-- src
#         |-- custom_msgs
USER_ROS2_INTERFACE_INSTALLATION_PATH=/home/user/ros2_ws

# [REQUIRED]
# The current directory where this .env file and the whole provided ROXSIE workspace is located.
# It also contains an  directory for user provided config files and an  directory where the generated code is stored.
# Example: /home/${USER}/roxsie_workspace
USER_ROXSIE_WORKSPACE_PATH=/home/user/roxsie_workspace

```

Afterwards start ROXSIE with:

```bash
docker compose run --name ROXSIE --rm ROXSIE

```

### 3. Generate PLC code using the ROXSIE command line tool

In the container environment, start the generation of the PLC code by choosing a configuration .yaml file located within the Workspace input folder.

```bash
roxsie -c .yaml

```

If your configuration is valid the output should look similar to this:

```bash
$ roxsie -c config.yaml
####################################  ROXSIE  ###################################
#  Version: 0.3.1                                                               #
#  Date:    dd.mm.YYYY                                                          #
#  Time:    HH:MM:SS                                                            #
#  Copyright (c) Siemens AG, 2026, and licensors. All rights reserved.          #
#  Portions include Open Source Software. See ReadMe_OSS_0.3.html for details.  #
#################################################################################

Gathering ROS 2 message types information ...
Generating PLC code ...
Please import the generated PLC code into your TIA project, export the TIA project .psc file, and place into the ROXSIE input folder. 
Please provide the file name of the .psc in the prompt when requested, e.g., 'tia_project.psc' and press ENTER to continue the generation process or 'n' and ENTER to exit the program.

```

Inside the output folder of your ROXSIE workspace a time-stamped folder has been created, that contains the generated PLC code. The path to the generated PLC sources is similar to the following:

```bash
workspace/output/time-stamp/output/plc_code/_roxsie_generated_rib_scl_structs.scl

```

### 4. Move generated PLC code to TIA V20 engineering machine

Now move the generated PLC source file `_roxsie_generated_rib_scl_structs.scl` to your TIA engineering machine. If you use a virtual machine you can use shared folders for example.

### 5. Import generated PLC source as external source file

Open the TIA project of your IPC or OpenController that shall establish a connection to ROS 2. Select the Software-Controller of the device and import the previously generated PLC source file into your TIA V20 project.

### 6. Generate blocks from external PLC source file

The imported source file can now be used to generate the PLC program blocks of the ROS 2 \<-> PLC connection:

There will be a warning message informing you that any duplicate blocks will be overwritten. Make sure to have a backup of your project and acknowledge the warning dialog.

Afterwards the following blocks are generated into your TIA project:

And the following UDTs representing the ROS 2 msg types inside your TIA portal:

### 7. Create .psc file of the device containing the ROXSIE generated blocks

To proceed with the generation of the ROS 2 package of the ROS 2 \<-> PLC connection a .psc file has to be created from your device inside the TIA project.

This can be done by pressing on Project/Memory card file/New/PC system configuration file:

Select the name of the .psc file.\
⚡ Do not use whitespaces inside the filename as this file will be used in a Linux-based OS in another step of the generation process.

Load your OpenController or IPC device into the .psc file by dragging and dropping the device onto the created Memory Card file:

A dialog will open. Press on load to load the device configuration into the .psc file.

### 8. Move .psc file to ROXSIE input directory

Now copy the created .psc file into the input directory of your ROXSIE workspace. If you are using a virtual machine you can use shared folders, or if using WSL, you can directly save into your WSL filesystem (normally found under \\wsl$\\Ubuntu)

### 9. Use .psc file to continue ROXSIE process to generate ROS 2 package

As the .psc file is now located in the ROXSIE context, you can now proceed with the generation of the ros2 package. The ROXSIE cli tool should still be waiting for the .psc file, at the state of the end of [Step 3](#3-generate-plc-code-using-the-roxsie-command-line-tool). Type in the name of the .psc file you created and press enter.

```bash
$ roxsie -c config.yaml
####################################  ROXSIE  ###################################
#  Version: 0.3.1                                                               #
#  Date:    dd.mm.YYYY                                                          #
#  Time:    HH:MM:SS                                                            #
#  Copyright (c) Siemens AG, 2026, and licensors. All rights reserved.          #
#  Portions include Open Source Software. See ReadMe_OSS_0.3.html for details.  #
#################################################################################

Gathering ROS 2 message types information ...
Generating PLC code ...
Please import the generated PLC code into your TIA project, export the TIA project .psc file, and place into the ROXSIE input folder. 
Please provide the file name of the .psc in the prompt when requested, e.g., 'tia_project.psc' and press ENTER to continue the generation process or 'n' and ENTER to exit the program.
plc-system-config.psc

```

If the generation was successful, the output of the cli tool should look similar to this:

```bash
$ roxsie -c config.yaml
####################################  ROXSIE  ###################################
#  Version: 0.3.1                                                               #
#  Date:    dd.mm.YYYY                                                          #
#  Time:    HH:MM:SS                                                            #
#  Copyright (c) Siemens AG, 2026, and licensors. All rights reserved.          #
#  Portions include Open Source Software. See ReadMe_OSS_0.3.html for details.  #
#################################################################################

Gathering ROS 2 message types information ...
Generating PLC code ...
Please import the generated PLC code into your TIA project, export the TIA project .psc file, and place into the ROXSIE input folder. 
Please provide the file name of the .psc in the prompt when requested, e.g., 'tia_project.psc' and press ENTER to continue the generation process or 'n' and ENTER to exit the program.
plc-system-config.psc
User Input: plc-system-config.psc
TIA project .psc file  found in input workspace. Continuing generation process ...
Generating ROXSIE API library and ROS 2 package ...
Successfully generated ROS 2 package and PLC code.

```

### 10. Move generated ROS 2 package to your desired workspace and build the package

Congratulations the generation process is now done. You can find the generated ROS 2 package in the output folder of your ROXSIE workspace under:

```bash
output//output/ros2_deployment/ros2_ws/src/ros2_package_name

```

Additionally, in the same output folder you will find the configuration .yaml file that was used for the creation of the ROS 2 package, as well as logs from the creation process in case of errors.

Please be sure to exit the ROXSIE container before continuing the process of moving your generated package. Otherwise some problems might arise while moving files around. To exit the container simply type `exit` into the command line.

#### a. Creating a new container based ROS 2 Workspace

Within the `output/ros2_deployment` folder, a Dockerfile, docker compose file and build script were also automatically generated. These files can be used to build a docker image as archive which can then be loaded into your runtime system.

Navigate to the ros2_deployment folder and run the following command to build the generated ROS 2 package as a deployable image:

```bash
# build_deploy_container.sh parameters:
# ros2-version: ROS 2 version [supported: humble, jazzy, kilted], default: jazzy
# container:    Container name, default: roxsie_bridge_example
# rmw:          RMW implementation [supported: fastdds, cyclonedds, zenoh], default: fastdds
bash build_deploy_container.sh --ros2-version jazzy --container roxsie_bridge_example --rmw zenoh

```

The script will build a docker image based on the standard ROS 2 image you have selected e.g. Jazzy, then compile the ROS 2 packages and setting up the required RIB runtime dependencies for the generated ROXSIE node. Afterwards, the `docker-compose.yaml` file will be updated to use the generated ROS 2 package.\
Finally, the script will export the generated docker image as a .tar archive with the name `_.tar`.

You can then follow these steps to deploy the generated image archive into your runtime system:

Copy the docker image archive to the Industrial OS.

```bash
scp roxsie_bridge_example_jazzy.tar [USER]@[DESTINATION_HOST_IP]:roxsie_bridge_example_jazzy.tar

```

Copy the docker-compose.yaml file to the Industrial OS.

```bash
scp docker-compose.yaml [USER]@[DESTINATION_HOST_IP]:docker-compose.yaml

```

Load the image again in the Industrial OS.

```bash
ssh [USER]@[DESTINATION_HOST_IP]
docker load < roxsie_bridge_example_jazzy.tar 

```

Now you are ready to run the generated ROS 2 package on your IPC via the generated `docker-compose.yaml` and docker compose.

#### b. Working on an existing ROS 2 Workspace

Although using the deploy process of docker images is recommended, you can use your existing ROS 2 workspace within Industrial OS. Therefore, all that is needed is to copy the complete ROS 2 package directory to your IPC runtime system's ROS 2 workspace and compile it via `colcon build`.

## Running the Generated Interface

### 1. Start RIB_App

Start the `RIB_App` executable inside the Industrial OS of your IPC device.

```bash
RIB_App

```

This application serves as a RIB participant broker like shown in the following illustration:

### 2. Load PLC program to the Software PLC

If not done already, load the TIA project that was extended by the ROXSIE generated program blocks to the Software PLC.\
The generated state machine will then try to automatically connect to the `RIB_App`.

### 3. Start ROS 2 Node

Start the ROS 2 node using the generated `docker-compose.yaml` file in your Industrial OS:

```bash
ssh [USER]@[DESTINATION_HOST_IP]
docker compose up

```

# Command Line Interface

## Required Argument

### YAML Configuration File

The only required parameter is the `--config` or the short version `-c` to provide a predefined YAML configuration file to ROXSIE for code generation.\
To run ROXSIE with a predefined YAML and start the whole generation process, run the following command:

```bash
roxsie --config file_name.yaml

```

Example console output:

```bash
$ roxsie -c example_config.yaml
####################################  ROXSIE  ###################################
#  Version: 0.3.1                                                               #
#  Date:    dd.mm.YYYY                                                          #
#  Time:    HH:MM:SS                                                            #
#  Copyright (c) Siemens AG, 2026, and licensors. All rights reserved.          #
#  Portions include Open Source Software. See ReadMe_OSS_0.3.html for details.  #
#################################################################################

Gathering ROS 2 message types information ...
Generating PLC code ...
Please import the generated PLC code into your TIA project, export the TIA project .psc file, and place into the ROXSIE input folder. 
Please provide the file name of the .psc in the prompt when requested, e.g., 'tia_project.psc' and press ENTER to continue the generation process or 'n' and ENTER to exit the program.

```

## Optional Arguments

### Verify YAML Config File

The `--verify` parameter, short version `-v`, can be set to check, if the provided YAML config meets the requirements.\
To verify, if the provided YAML config matches the required syntax, run the following command:

```bash
roxsie --verify --config file_name.yaml

```

Example console output:

```bash
$ roxsie -v -c example_config.yaml
####################################  ROXSIE  ###################################
#  Version: 0.3.1                                                               #
#  Date:    dd.mm.YYYY                                                          #
#  Time:    HH:MM:SS                                                            #
#  Copyright (c) Siemens AG, 2026, and licensors. All rights reserved.          #
#  Portions include Open Source Software. See ReadMe_OSS_0.3.html for details.  #
#################################################################################

YAML verifying and parsing successful.

```

### Disable User Input Wizard

The `--non_interactive` parameter, short version `-y`, can be used to skip all user interactions and process the whole generation process automatically.\
To start the whole generation process non-interactively, additionally the [--psc_file](#psc-file) argument needs to be provided:

```bash
roxsie --non_interactive --config file_name.yaml --psc_file tia_project.psc

```

Example console output:

```bash
$ roxsie -y -c example_config.yaml -t example_tia_project.psc
####################################  ROXSIE  ###################################
#  Version: 0.3.1                                                               #
#  Date:    dd.mm.YYYY                                                          #
#  Time:    HH:MM:SS                                                            #
#  Copyright (c) Siemens AG, 2026, and licensors. All rights reserved.          #
#  Portions include Open Source Software. See ReadMe_OSS_0.3.html for details.  #
#################################################################################

Gathering ROS 2 message types information ...
Generating PLC code ...
Generating ROXSIE API library and ROS 2 package ...
Successfully generated ROS 2 package and PLC code.

```

### PSC File

The `--psc_file` parameter, short version `-t`, can be used to process all code generation steps without user interaction.\
It will provide the TIA Portal generated PC System Configuration (.psc) file for internal struct comparison against the PLC based data type definitions.\
⚠️ This argument needs to be combined with the [--non_interactive](#disable-user-input-wizard) argument.

Example usage (creates the same command line output as in the section [Disable User Input Wizard](#disable-user-input-wizard)):

```bash
roxsie --non_interactive --config file_name.yaml --psc_file tia_project.psc

```

### Usage Help

Calling ROXSIE without any arguments or with the `--help` argument, short version `-h`, will print command line usage information:

```bash
roxsie --help

```

Example console output:

```bash
$ roxsie -h
####################################  ROXSIE  ###################################
#  Version: 0.3.1                                                               #
#  Date:    dd.mm.YYYY                                                          #
#  Time:    HH:MM:SS                                                            #
#  Copyright (c) Siemens AG, 2026, and licensors. All rights reserved.          #
#  Portions include Open Source Software. See ReadMe_OSS_0.3.html for details.  #
#################################################################################

Command line parameters: 
  --config [-c]           YAML configuration file name, e.g., 'config.yaml' which defines system specifications (required)
  --help [-h]             Get information about the ROXSIE usage and CLI parameters
  --non_interactive [-y]  Disable the wizard mode and run all stages without user interaction (optional)
  --psc_file [-t]         TIA project .psc file name, e.g., 'tia_project.psc' (optional)
  --verify [-v]           Verification of the provided YAML configuration file (optional)

```

# YAML Configuration Structure

## Introduction

As ROXSIE provides a command line based interface for users, a service was needed to give users the possibility to specify their system using a [YAML](https://yaml.org) based configuration file.\
In this configuration, users can declare the [system](#system), which stands for the ROS 2 package that will be the wrapper for all defined data structures. Additionally, users shall define the [data structure connections](#topic) based on ROS 2 topics.

## System

An example of a system definition is shown below.\
The generated package will setup a fully functional ROS 2 package, that works out out of the box by using the parameters specified below.

```yaml
system:
  ros2:
    package_name: example_package
    node_name: example_node
  rib:
    RIB_App_IPv4: 127.0.0.1
    RIB_App_port: 27567
    RIB_Version: "v2.2.4"
  # optional
  plc:
    # optional
    rib_config_db_name: "rib_config_topic_DB"
    rib_connect_state_machine_fc_name: "rib_connect_build_up_FC"

```

### `system (required)`

The `system` keyword specifies the content of the generated ROS 2 package.

```yaml
system:

```

### `ros2 (required)`

The `ros2` keyword declares all content that is ROS 2 specific.

```yaml
  ros2:

```

### `package_name (required)`

The `package_name` keyword is used to define the name for the generated ROS 2 package.

```yaml
    package_name: 

```

### `node_name (required)`

The `node_name` keyword is used to define the name for the generated ROS 2 node inside the package.

```yaml
    node_name: 

```

### `rib (required)`

The `rib` keyword defines the connection to the RIB_App, that is used to manage different shared memory connections of the RIB.

```yaml
  rib:

```

### `RIB_App_IPv4 (required)`

The `RIB_App_IPv4` keyword specifies the IPv4 address of the RIB_App, to which the RIB API will be connected to via TCP based connection.\
Valid values are either `localhost` or a correctly set IPv4 address e.g. `127.0.0.1`.\
Note: The value can later be changed in the source code.

```yaml
    RIB_App_IPv4: localhost

```

### `RIB_App_port (required)`

The `RIB_App_port` keyword is used to define the TCP based port, at which the RIB_App is reachable.\
Valid are full numbers e.g. `80`.\
Note: The value can later be changed in the source code.

```yaml
    RIB_App_port: 

```

### `RIB_Version (required)`

The `RIB_Version` keyword specifies the RIB version which is being used by the system, e.g. `v2.2.4`.\
In order to get the RIB version currently installed in your IPC, the [command described in the FAQ section](../../troubleshooting.html#how-can-i-check-which-rib-version-is-installed-on-my-ipc) can be used.

```yaml
    RIB_Version: 

```

### `plc (optional)`

The `plc` keyword declares all content that is PLC specific.

```yaml
  plc:

```

### `rib_config_db_name (optional)`

The `rib_config_db_name` keyword can be used to change the name of the data block associated to the connection control and status of the RIB. The default value is "RIB_CONFIG_DB".

```yaml
    rib_config_db_name: 

```

### `rib_connect_state_machine_fc_name (optional)`

The `rib_connect_state_machine_fc_name` keyword can be used to change the name of the function block handling the automatic connection build up to the RIB. The default value is "RIB_CONNECT_STATE_MACHINE_FC".

```yaml
    rib_connect_state_machine_fc_name: 

```

## Topic

The `Topic` area relies on the principles of the [ROS 2 topic system](https://docs.ros.org/en/jazzy/Tutorials/Beginner-CLI-Tools/Understanding-ROS2-Topics/Understanding-ROS2-Topics.html) and is divided into two possible connection directions, either from [ROS 2 to PLC](#ros-2-to-plc) or from [PLC to ROS 2](#plc-to-ros-2).\
Those connections are based on the data structures that are defined within them.

### ROS 2 to PLC

To define a ROS 2 subscriber that writes its received data to the PLC, the `ros2_to_plc` tag shall be used.\
Within it, all needed data structures can be specified. Each type specification will be used to create a corresponding subscriber within the generated ROS 2 package.\
An example is provided below.

```yaml
ros2_to_plc:
  topics:
    - type: "custom_msgs/msg/Type.msg"
      ros2_topic: "/topic"
      rate: 100.0
    - type: "custom_msgs/msg/AnotherType.msg"
      ros2_topic: "/another_topic"
      rate: 5.0

```

#### `ros2_to_plc (required)`

The `ros2_to_plc` keyword defines the connection direction from ROS 2 to the PLC.

```yaml
ros2_to_plc:

```

#### `topics (required)`

The `topics` keyword is used to clarify, that the created ROS 2 part will be a subscriber, that listens on a specified topic.

```yaml
  topics:

```

#### `type (required)`

The `type` keyword defines the ROS 2 based topic message type, that will be searched for either in the provided custom interface path or the ROS 2 standard installation path.

```yaml
    - type: "/msg/.msg"

```

✔️ Valid values are:

- In case of a standard ROS 2 based message type e.g. geometry_msgs/msg/Twist located at the path specified via `USER_ROS2_INSTALLATION_PATH` ENV variable:\
  `"geometry_msgs/msg/Twist.msg"`
- In case of a custom message type from a custom interface package located at the path specified via `USER_ROS2_INTERFACE_INSTALLATION_PATH` ENV variable:\
  `"custom_msgs/msg/Type.msg"`

❌ Wrong values are e.g.:

- `"geometry_msgs/msg/Twist"`
- `"geometry_msgs/Twist"`

#### `ros2_topic (required)`

The `ros2_topic` keyword defines the ROS 2 topic name of the desired message type, where the '/' is optional.\
⚠️ The name must be unique in the YAML configuration!

```yaml
      ros2_topic: "/"

```

ROS 2 topic namespaces can be provided e.g.:

```yaml
      ros2_topic: "robot_1/topic_name"

```

#### `rate (required)`

The `rate` keyword is used to define the rate in Hz, in which messages will be written to the PLC on the specified topic.\
It is used to configure the interrupt cycle of the PLC side for this topic.

```yaml
      rate: 10.0 # in Hz

```

✔️ Valid subscription rates [Hz] are (as floating point values):

- All values > 0.0
- All values \<= 1000.0

⚠️ Values must not match the exact real subscription rate, but one has to keep in mind, that each generated PLC interrupt OB will provide a cycle time with the same rate as specified in the YAML!

### PLC to ROS 2

To define a ROS 2 publisher that reads the data received from the PLC and publishes it to ROS 2, the `plc_to_ros2` tag shall be used.\
Within it, all needed data structures can be specified similar to the configurations in the section [ROS 2 to PLC](#ros-2-to-plc). Each type specification will be used to create a corresponding publisher within the generated ROS 2 package.\
An example is provided below.

```yaml
plc_to_ros2:
  topics:
    - type: "custom_msgs/msg/Type.msg"
      ros2_topic: "/topic"
      rate: 250.0
    - type: "sensor_msgs/msg/BatteryState.msg"
      ros2_topic: "/battery"
      rate: 1.0

```

#### `plc_to_ros2 (required)`

The `plc_to_ros2` keyword defines the connection direction from the PLC to ROS 2.

```yaml
plc_to_ros2:

```

#### `topics (required)`

The `topics` keyword is used to clarify, that the created ROS 2 part will be a publisher, that publishes PLC data on a specified topic.

```yaml
  topics:

```

#### `type (required)`

The `type` keyword defines the ROS 2 based topic message type, that will be searched for either in the provided custom interface path or the ROS 2 standard installation path.

```yaml
    - type: "/msg/.msg"

```

✔️ Valid values are:

- In case of a standard ROS 2 based message type e.g. geometry_msgs/msg/Twist located at the path specified via `USER_ROS2_INSTALLATION_PATH` ENV variable:\
  `"geometry_msgs/msg/Twist.msg"`
- In case of a custom message type from a custom interface package located at the path specified via `USER_ROS2_INTERFACE_INSTALLATION_PATH` ENV variable:\
  `"custom_msgs/msg/Type.msg"`

❌ Wrong values are e.g.:

- `"geometry_msgs/msg/Twist"`
- `"geometry_msgs/Twist"`

#### `ros2_topic (required)`

The `ros2_topic` keyword defines the ROS 2 topic name of the desired message type, where the '/' is optional.\
⚠️ The name must be unique in the YAML configuration!

```yaml
      ros2_topic: "/"

```

ROS 2 topic namespaces can be provided e.g.:

```yaml
      ros2_topic: "robot_1/topic_name"

```

#### `rate (required)`

The `rate` keyword is used to define the rate in Hz, in which messages will be read from the PLC on the specified topic.\
It is used to configure the interrupt cycle of the PLC side as well as a ROS 2 based timer callback to read the data at the provided rate.

```yaml
      rate: 10.0 # in Hz

```

✔️ Valid subscription rates [Hz] are (as floating point values):

- All values > 0.0
- All values \<= 1000.0

⚠️ Values must not match the exact real subscription rate, but one has to keep in mind, that each generated PLC interrupt OB will provide a cycle time with the same rate as specified in the YAML!

## Example

Based on provided message type definitions in a custom `interface_package`, a ROXSIE YAML configuration `config.yaml` could look like the following:

```yaml
system:
  ros2:
    package_name: roxsie_ros2_package
    node_name: roxsie_ros2_node
  rib:
    RIB_App_IPv4: 127.0.0.1
    RIB_App_port: 27567
    RIB_Version: "v2.2.4"

ros2_to_plc:
  topics:
    - type: "interface_package/msg/Type.msg"
      ros2_topic: "/topic"
      rate: 50.0
    - type: "interface_package/msg/AnotherType.msg"
      ros2_topic: "/another_topic"
      rate: 10.0
    - type: "gemetry_msgs/msg/Twist.msg"
      ros2_topic: "cmd_vel"
      rate: 100.0

plc_to_ros2:
  topics:
    - type: "interface_package/msg/Type2.msg"
      ros2_topic: "namespaced/topic2"
      rate: 250.0
    - type: "geometry_msgs/msg/PoseStamped.msg"
      ros2_topic: "/pose"
      rate: 50.0

```
# SITRANS SCM IQ documentation

# SITRANS SCM IQ API reference

Find descriptions of different standard Industrial IoT APIs and services and their current usage within SITRANS SCM IQ as well as ideas for your potential reuse.

## Asset Management Service

Asset Management Service supports you in creating digital representations of your physical assets. Such assets could be pumps, engines, gas turbines, conveyer belts.

SITRANS SCM IQ provides the following data points via Asset Type SITRANSMS to the Asset Management Service:

With this API you could possibly combine IoT assets with your “classical” plant hardware in your plant lifecycle software.

## Event Management API

There are many devices that are connected to [Insights Hub](../insights-hub/overview.html). Sometimes, devices show an unexpected behavior (for example, leakage in a motor pump or freezer exceeding the temperature limit) which demands special attention. These situations are known as events which can be digitally represented in Insights Hub. For SITRANS SCM IQ all detected anomalies in will be injected as events. Fetch all anomalies, their type (info, warning, error) and the time stamp of each event. To activate the event API you need to buy “Events Upgrade”, to be found in the upgrade catalogue from [Insights Hub](../insights-hub/overview.html). What could you use this data for? Examples could be to trigger maintenance ticket in your order management system, or to analyze overall reliability of different assets.

## IoT File Service

The IoT File Service provides the user with file management for files related to assets. The service provides an interface to manage data, search files and download them to specified agents. Files can also be uploaded to a temporary storage and managed there. SITRANS SCM IQ offers to upload videos, pictures and documents per anomaly and asset. These are uploaded during the anomaly handling in SITRANS Anomaly IQ and stored in the IoT File Storage.

## IoT Time Series Service

The IoT Time Series Service is used to create, read, update, and delete time series data. Time series data is stored against an asset and an aspect. A time series record consists of a timestamp, one or more values, and an optional quality indicator for each variable, which is defined by the aspect type. SITRANS SCM IQ provides the following data points via Asset Type SITRANSMS:

- You need an activated anomaly prediction model to store variable “anomaly_status” (possible values 20, 30, 40) and “deviation_index”.
- “acceleration_XYZ” data of SITRANS MS200 is provided as peak-to-peak value in the frequency band 13Hz to 3300Hz.

It automatically detects whether the monitored asset is running or not, and gives back a value for it:

- 0 = not running
- 1 = running

What could you possibly do with this data? There are endless possibilities to create value with SITRANS SCM IQ time series data. Ideas could be to combine it with power consumption to calculate asset efficiency, or to combine with process data to monitor process health.

# Contact

For any inquiries, whether organizational matters like the procurement of API Credentials or technical questions about SITRANS SCM IQ, please direct your communication to our Industry Online Support: [support.industry.siemens.com](https://support.industry.siemens.com/my/de/en/).

Find the available information under "Product Support" > “SCM IQ” quickly and at any time.\
Please raise a “Support Request” > “SCM IQ” for getting answers to your specific needs. If you have not logged in before, you need to register: free, simple and efficiently. Our support will get back to you in short time.

# Getting started with our application

SITRANS SCM IQ utilizes standard Industrial IoT APIs and services to supply data. To make use of this data, you need access to an instance of SITRANS SCM.\
If you only want to use machine data analysis then the SITRANS SCM IQ app is sufficient. If you want to make use of the specialized vibration analysis features, you need to connect at least one SITRANS MS200 via one SITRANS CC220.

Please find our Getting Started Guide of the [SITRANS SCM IQ](https://support.industry.siemens.com/cs/mdm/109808032?c=150219847435&lc=en-DE).

There is also a [YouTube playlist with comprehensive online tutorials](https://www.youtube.com/playlist?list=PLw7lLwXw4H51PBRRUC3_4ajiYq4y5IRLb).

## Demo Video

- [Overview](overview.html)
- [Getting Started](getting-started.html)
- [API Reference](api.html)
- [Contact](contact.html)

# SITRANS SCM IQ

## Overview

With SITRANS SCM IQ system you can monitor the condition of your plant's mechanical components, like pumps, compressors, or gearboxes from anywhere, 24-7. For this you can use specialized wireless SITRANS MS200 multi-sensors seamlessly connected to Industrial IoT via the industrial gateway SITRANS CC220. Additionally you can inject any existing machinery data into SITRANS SCM IQ machine learning algorithms for anomaly detection.

The system reliably issues warnings of potential asset failures in due time and thus prevents unexpected plant downtime.

Unlock the full potential of SITRANS SCM IQ powerful algorithms and experience the convenience of an out-of-the-box AI solution for increasing uptime and securing your business's future success.

## How everyone benefits from SITRANS SCM IQ

### Out-of-the-box artificial intelligence

SITRANS SCM IQ uses two types of anomaly detection algorithms to provide reliable condition monitoring. One algorithm is specialized on consuming the MS200 vibration data to monitor rotating equipment, the other algorithm is more generic an can consume any time-stamped machine data. SITRANS SCM IQ is generally designed as a non-expert system and can be used without knowledge on data science or AI.

### Start small, scale up

SITRANS SCM IQ has a low initial invest and is also suitable for customers with low digitalization level. The system is easy retrofit and can be used in any industry. You can start with monitoring single assets, easily scaling to complex production lines or plant wide deployment.

### Increase uptime, improve maintenance processes, and maximize asset utilization

SITRANS SCM IQ not only helps to keep your plant running smoothly, it also reduces the need for costly repairs and time-consuming maintenance. Choose SITRANS SCM IQ today and benefit from asset insights that have been hidden so far. With its easy-to-use interface and comprehensive features, SITRANS SCM IQ is the smart choice for any industrial plant.

## Key Features

### Provide accurate asset data

SITRANS SCM IQs’ AI continuously analyses the vibration characteristics of your rotating equipment and sends you a notification when an abnormal condition is detected. You and your team can rely on automated condition monitoring and optimize your maintenance schedules accordingly.

The artificial intelligence can furthermore consume any machinery data (e. g. power consumption, rotating speed, operating mode) for anomaly detection.

Benefit from reliable monitoring and intuitive visualization of asset, sensor and diagnostic data.

### Gain insights into asset performance

Administrate and document asset anomalies and machine outages for immediate distribution within your team - gain a deep insight into each asset performance and its weak spots. Count working hours from formerly unsupervised assets and stay ahead of the game.

### Make insightful decisions

With a reliable, almost real-time monitoring SITRANS SCM IQ provides meaningful insights to make informed, data-driven decisions faster.

Knowing the condition of the mechanical assets in your process plant 24-7 gives you a clear advantage, because you can identify imminent device failures in good time before an incident occurs. Plan your maintenance activities instead of enduring costly downtimes and conducting ad-hoc measures. Keep your machines healthy without under or over servicing.
# Teamcenter (X) Product Cost Management documentation

# Contact

For any technical questions related to the TcPCM REST API, please contact us at:

## Support Center (preferred)



Note

Choose product *Teamcenter Product Cost Management*

## Phone

Find worldwide Customer Support Helpdesk Line phone numbers here:



## Web

[Teamcenter (X) Product Cost Management](https://plm.sw.siemens.com/en-US/teamcenter/solutions/product-cost-management/)

## Community

[TcPCM Community](https://community.sw.siemens.com/s/topic/0TO4O000000ZINTWA4/plm-product-cost-management)

# Frequently asked questions

## I can log in, but can not get a response in other messages

Please make sure you are prepending “Bearer “ string in front of the access token in subsequent messages (some clients do that automatically, so in such cases avoid doubling this string).

## I get an HTTP error

Please read the description of the error codes assigned to the message in swagger (example below) and make sure that the user has the right “Manage REST-API” assigned in TcPCM User administration.

## Is a special license required?

For Calculation export and Tool calculation export the license - Bulk Export API of Dynamic Calculations [PPC10029] is required.

## Where can I find the export or import definition id?

In the export/import definition click edit and find the Cross-database identifier(GUID).

## How can I find out, which calculations are a version of each other?

Add "VersionStream" column to your export definition. Each version of the calculation shares the same "VersionStream" string.

Note

Please, check the availability of "VersionStream" column in your export definitions. It is not available in TcPCM versions released before the start of 2022.

## Are there more examples available?

At the moment, PCM R&D provides additional resources upon request:

- Excel example with connection done with VBA
- Python example
- Postman example

## Can I get installation assistance?

If you require assistance with the installation or configuration of Teamcenter Product Cost Management, Customer Support will guide you to the DI SW Consulting team who can help you with this.

Note

This may result in additional costs.

# Getting Started

## Introduction

Every three-tier TcPCM installation runs a REST server, that can be used to access data in TcPCM without the need of the rich client. It can be used to import data as well.

## REST API Setup

### License

Endpoints (features) that export data require purchasing the license:

> PPC10029 - Bulk Export API of Dynamic Calculations

Other API endpoints do **not** require a specific license.

### User rights

To be able to interact with the REST API, the users need to have the “**Manage REST-API**” permission granted through a role assignment with such permission.

### Add client information

One part of the security concept is the configuration of API clients (not users) that are allowed to interact with the REST API.

It is recommended to create separate clients for major versions of connection applications, so that - in case of need – obsolete versions can be prevented from connecting.

#### TcPCM 2412 and newer

Add a client via "Administration -> Manage Rest API Clients".

After adding a client you need to store the secret safely on your client for a valid authentication.

#### 2406 and before

In the database is a table which is called “TcPCM.OAuthClients”. The following image shows a sample record for a client called “TcPCM” with secret “A980XXXX- XXXX-XXXX-XXXX-XXXXXXXXXXXX”.

## General concepts

The REST API follows the principle of "Request -> Response". Currently, TcPCM doesn't support long-running/asynchronous calls.

## Testing the REST API

The REST API can be tested via the provided Postman query collection or any other REST API capable tool.

### Ping

The easiest way to test the connection to the API is to use a `ping` call. It is a quick response so that the developer can verify that the server is running and the connection to the API works.

## Authentication

### Authentication using TcPCM already existing authentication methods

TcPCM supports three different login flows:

1. **Resource Owner Password flow**: Use the `username` and `password` with the `grant_type` *password* and `authenticatorId` *Siemens.TCPCM.AuthenticationMethod.TcPCM* or *Siemens.TCPCM.AuthenticationMethod.Windows*.
1. **Token Exchange flow**: Use the `subject_token_type` and `subject_token` with the `grant_type` *urn:ietf:params:oauth:grant-type:token-exchange* and `authenticatorId` *Siemens.TCPCM.AuthenticationMethod.SamAuth*.
1. **Custom flow**: Use the `tcpcmSessionKey` with the `grant_type` *applicationSession* without `authenticatorId`.

Authentication is done by calling the `/token` endpoint. The `client_id` and `client_secret` must be passed as described in the client setup section.

In a successful response, the `access_token` (also called bearer token) is needed for subsequent calls to the REST API endpoints.

The `refresh_token` can be used to obtain a new access token after the expiration of the access token.

To do that invoke a `/token` with parameter `request_token` instead of username and password. The `client_id` and `client_secret` are still required.

The `refresh_token` is also a parameter that is needed to perform a logout.

### Authentication using certificate

Setting up the environment for using certificate authentication is more complicated and it is not provided out of the box, see [Securing with certificates](securing-with-certificates.html).

### Logout

It is possible to end the validity of access and refresh tokens by calling `/Account/Logout` and passing the `refresh_token` and `access_token` as parameters.

## General message structure

Standard messages are protected by using the bearer token authentication.

### Request

To perform a request, a header and a body (except `GET` endpoints) must be provided.

In the header, the authentication token is provided in the format of “Bearer AccessToken” in the `Authorization` field.

Please note, the **space** between `Bearer` and the `access_token`. If `/token` returns an access token such as *ABCD* your subsequent messages need to have the header parameter in the format “Bearer *ABCD*”. Some REST clients will do that automatically (this is the case if you are using Postman) and some do not. Pay attention to this parameter, since it is a source of common problems.

### Calculation/Import Request

To import calculation data the method Calculations/Import must be called with the parameter

- **Data** a JArray of calculation data to import
- **Guid** of an import configuration that is stored in the TcPCM server database
- **TargetType** to define the target parent of the calculations to import (Folder, Project)
- **TargetId** ID of the target object

### Response

The response structure depends on the message and configuration.

As an example: The response of a calculation/Export call will be a result container in the form:

After a successful export, an example of data is

Note

The order of returned calculation is not guaranteed. It is strongly recommended adding the calculationID (database identifier) as one of the exported fields. That enables pairing of records between request and response (not necessary if you are just requesting one calculation).

- [Overview](overview.html)
- [Getting Started](getting-started.html)
- [Securing with certificates](securing-with-certificates.html)
- [FAQ](faq.html)
- [REST API](rest-api-spec.html)
- [Versions](versions.html)
- [Contact](contact.html)

# Overview

## The Gamechanger

## Cost and CO2

## Collaboration with your global supply chain

Removing the barriers to increase productivity and reduce risk

## Integration

Integrating all data available at the early stages of development allows the realization of the digital twin of cost.

TcPCM is **open for extensions** to address custom use cases and enable integration with third-party systems while helping to ensure data integrity and security.

## Roll up

Bottom-up model to roll up cost and CO2e emissions along the value chain.

# TcPCM REST API

- [Download OpenAPI Specification](tcpcm-api-spec.yaml)

# Securing with certificates

## Introduction

This is a very rough guide. For more details, there is a lot of [help material provided by Microsoft](https://docs.microsoft.com/en-us/iis/configuration/system.webserver/security/authentication/iisclientcertificatemappingauthentication/).

## Install IIS Client Certificate Mapping feature

To use the certificate mapping the feature needs to be installed.

## Configuring Web Page for REST-API

To enable certificate mapping the REST-API needs to be installed in its own web site. This is necessary because all authentication methods need to be disabled. The TcPCM rich client is using a SOAP based interface (Windows Communication Foundation) that needs anonymous authentication on the IIS level. Because the soap interface is checking for certificates on a deeper level.

### SSL Settings

### Disable all authentication methods

### Enable certificate mapping

After disabling all authentication methods the certificate mapping needs to be enabled in the configuration editor.

## Version Matrix

| TcPCM                                 | API  |
| ------------------------------------- | ---- |
| 2606 and newer                        | v1.5 |
| 2512.2602 and subsequent 2512 updates | V1.4 |
| 2512                                  | V1.3 |
| 2412 and 2506                         | V1.2 |
| 2406.0001 and subsequent 2406 updates | V1.1 |
| 2406 and before                       | V1.0 |

## Changelog

### V1.5 (latest)

- [Download V1.5](tcpcm-api-spec.yaml) / [Show](rest-api-spec.html)
- Added the "material detail GUID" (`data[].materialHeaders[].validityParameters[].materialPrices[].materialDetailGuid`) to the result of the `/api/v1/projects/{id}/project-assumptions` endpoint
- Added `Guids` to the request body of the `/api/v1/MasterData/Export` endpoint to request master data with GUIDs instead of database identifiers

### V1.4

- [Download V1.4](archive/tcpcm-api-specV1.4.yaml)
- Added new endpoint `/api/v1/aggregated-profitability-calculation-report` for exporting aggregated profitability calculations
- Added `DELETE` method to the `/api/v1/projects/{id}` endpoint to support deletion of projects
- Added new endpoint `/api/v1/product-costing/parts/{id}` to delete product costing parts

### V1.3

- [Download V1.3](archive/tcpcm-api-specV1.3.yaml)
- Added new endpoint `/api/v1/projects/{id}` for obtaining project data
- Added new endpoint `/api/v1/projects/{id}/project-assumptions` for obtaining the project assumptions of a project

### V1.2

- [Download V1.2](archive/tcpcm-api-specV1.2.yaml)
- Added new endpoint `/api/v1/administration/application-version` for obtaining the running TcPCM version
- Added "Token Exchange flow" to the `/token` endpoint for the possibility of using a third-party token (like SAMAuth token) to retrieve the bearer token
- Added optional parameter `ExportConfigurationGuid` in `api/v1/Calculations/Import` endpoint for exporting the imported BOM directly (e.g. exporting the calculated costs and carbon footprint)

### V1.1

- [Download V1.1](archive/tcpcm-api-specV1.1.yaml)
- Added new endpoint `api/v1/profitability-calculation-report` for exporting profitability calculations

### V1.0

- [Download V1.0](archive/tcpcm-api-specV1.0.yaml)
- Initial release
# Vilocify Vulnerability Services

# REST API v2

The documentation of the API v2 is available in an [interactive API Explorer](https://portal.vilocify.com/documentation).

If you are using Python, you might benefit from the [Vilocify Python SDK](https://github.com/siemens/vilocify-sdk-python) to interact with the API v2.

# Contact

- *Help desk*: 
- *Email*: [svm.ct@siemens.com](mailto:svm.ct@siemens.com)

# Vilocify

Vilocify alerts you about vulnerabilities affecting software and hardware utilized in your products and infrastructure. Our unique monitoring approach ensures that you receive the relevant information based on your individual selection of third-party components. We achieve this by monitoring thousands of security sources using advanced technologies like natural language processing in combination with in-depth analysis by acknowledged cybersecurity experts. Whenever relevant information is detected, it's analyzed and evaluated by our dedicated team. The processed information is then made available through the [Vilocify Portal](https://portal.vilocify.com), the [REST-API](apiv2.html), and email notifications.

## Monitoring infrastructure

The Vilocify service monitors and aggregates vulnerability information from thousands of different sources, like (but not limited to):

- official vendor advisory pages
- vulnerability databases
- national CERTs
- security researcher exchange platforms
- source code repositories

Our monitoring infrastructure fetches information from many source formats and types, no matter if the information is made available within a web page, API, RSS feed, mailing list, etc.

### Expert analysis

The information generated by the monitored sources is then evaluated by a team of security experts. The information is assessed, filtered, enriched, and normalized. This allows us to create standardized security notifications, containing (amongst others) a description, a severity rating, and a recommended action.

### Delivery channels

Vulnerability information is then made available via our [Portal](https://portal.vilocify.com/) or our [REST-API](apiv2.html).

## Vilocify service in figures

The Vilocify service in figures as of mid 2025:

- 2.000+ monitored sources
- 140000+ notifications
- 400.000+ components
- CVE completeness since 2021
- EUVD completeness since 2025

## Contact

Users of the Vilocify Service can send queries about content of Vilocify security notifications, technical problems or organizational issues to the Vilocify Team via opening a ticket in our [support portal](https://support.vilocify.com/hc/en-us). Support is available in English and German.

## Privacy policy

Vilocify stores only information required for the proper operation of the service.

- [Home](home.html)
- Service description
  - [Service description](service-description/index.html)
  - [Component](service-description/component.html)
  - [Component request](service-description/component-request.html)
  - [Security notification](service-description/security-notification.html)
  - [Monitoring list](service-description/monitoring-list.html)
  - [Notifications email](service-description/notifications-email.html)
  - [Membership and organization](service-description/membership-and-organization.html)
  - [Roles and permissions](service-description/roles-and-permissions.html)
  - [Consumption reporting](service-description/consumption-reporting.html)
  - [Glossary](service-description/glossary.html)
- Getting started
  - [Getting started](getting-started/index.html)
  - [Creating a monitoring list](getting-started/creating-a-monitoring-list.html)
  - [Managing monitoring lists](getting-started/managing-monitoring-lists.html)
  - [Requesting a new component](getting-started/requesting-a-new-component.html)
  - [Understanding a security notification](getting-started/understanding-a-security-notification.html)
- [API v2](apiv2.html)
- FAQ
  - [General Questions](faq/index.html)
  - [Vilocify portal emails](faq/faq-vilocify-portal-emails.html)
- [Contact](contact.html)

# Frequently asked questions

## What's Vilocify?

The Vilocify Service constantly monitors a broad amount of information sources to discover and report new security vulnerabilities and corresponding security patches for a wide range of software and hardware components. You'll find more information about Vilocify in its [Service Description](../service-description/index.html) page.

## I didn't find the component I'm looking for in Vilocify Portal. What am I expected to do?

First, use the component search available in the [Components](https://portal.vilocify.com/components) tab in Vilocify Portal and search for different names your component might be also known as. It may help to search only for the name of the component and leave out the vendor name. For example, don't type `github` but only the name of the GitHub project you are looking for. If you don't find the component at all or the specific version is missing use the [request form](https://portal.vilocify.com/components#newrequest) to request a new component. More information about component requests can be found in the [Requesting a New Component](../getting-started/requesting-a-new-component.html) page. It may take some days before your request is completed because every single request is manually verified. Once your component request is processed, you will be notified by email.

## Can I add a wildcard component to my monitoring list?

Yes, we do have wildcard components that allow you to monitor a wide number of versions without the need to specify each version separately. Consider the following examples:

- By adding [Debian 11.x](https://portal.vilocify.com/components#128970) to your monitoring list you will receive all security advisories for Debian 11, without having to specify single update levels.
- Adding [Apache Tomcat 8.x](https://portal.vilocify.com/components#30560) allows you to install security updates of your Tomcat server without having to continuously update your list.
- If you want to monitor Windows Server 2008 R2 regardless of its service pack level, you can use the component [Microsoft Windows Server 2008 R2 All Versions](https://portal.vilocify.com/components#19434).

## Can I monitor "All Versions" of a specific component?

Yes, in most cases all versions of a component can be monitored. To monitor all versions of a certain component you can request the component with version "All Versions" through the [request form](https://portal.vilocify.com/components#newrequest) in case it doesn't already exist in the Vilocify component catalog. "All Versions" components can reduce the effort of maintaining your monitoring lists, because you don't have to update the "All Versions" components on your list when new versions are used in your project. However, you might receive a larger number of less relevant notifications, since you will also receive notifications for versions that you might not be using. If the components on your monitoring list are relatively static, it might be better to use components in specific versions.

## Why are Linux distribution packages and OpenShift container images only available in "All Versions?"

Generally, distributions only support the latest version of packages, and packages are updated frequently. Additionally, due to software dependencies, updating single packages/images is often not possible. As a result, most users usually take an interest in the latest version of a distribution package or a container image. Therefore, using "All Versions" avoids having to constantly update your monitoring lists. Don't worry if your package/image component wasn't mapped to the specific version, you will still receive relevant security notifications.

Tip

Linux distribution packages are available (or can be requested) for a specific distribution version, for example `Debian 9 Package: bash`, `RHEL 8 Package: openssl`, `Ubuntu 22.04 Package: apache2`, etc.

OpenShift container images are available (or can be requested) for a specific distribution version, for example `OpenShift Container Image: rhel8/postgresql-12`, `OpenShift Container Image: lvms4/lvms-operator-bundle`, `OpenShift Container Image: odf4/odf-must-gather-rhel9`, etc.

If you are monitoring the generic distribution packages without a distribution version like *Debian Package: bash*, *RHEL Package: openssl*, or *Ubuntu Package: apache2*, expect to receive more irrelevant notifications.

## I want to receive security notifications for my Linux distribution. Do I need to add all the packages one by one?

No, there is no need to add single Linux packages to your monitoring list. For all the major versions, you should be able to find a component representing the distribution in its entirety. Examples for such components are:

- [Red Hat Enterprise Linux Server 6](https://portal.vilocify.com/components#14899)
- [SUSE Linux Enterprise Server 12](https://portal.vilocify.com/components#24762)
- [Oracle Linux 7](https://portal.vilocify.com/components#37288)
- [Debian GNU/Linux 9](https://portal.vilocify.com/components#31346)

Adding such components implies that you will receive all security advisories for the distribution, without having to specify single packages. If you can't find the desired Linux distribution, please feel free to [request a new component](https://portal.vilocify.com/components#newrequest).

## I don't have time to maintain a monitoring list. What are my options?

As we publish over 15000 notifications every year, we strongly recommend creating and properly maintain monitoring lists with the desired components, in order to be notified only about security issues relevant to you. With [usage of wildcard components](#can-i-add-a-wildcard-component-to-my-monitoring-list), the required efforts should be minimal. If nonetheless this option isn't viable for you, there area few possible alternatives:

**REST API**: For advanced use cases, we offer a REST API which allows to retrieve information about components, monitoring lists, and notifications. For more details as well as a thorough documentation, please consult the [REST API](../apiv2.html) page.

## Is there any kind of automation possible?

Yes, there is the [Vilocify REST API](../apiv2.html) available for automation.

## A component I need to monitor has security-relevant information only in a protected customer area. How does Vilocify deal with that?

Some vendors provide security information only to customers with active support accounts without making it available to the public. If you request a component (or add an existing one to a monitoring list) where this is the case, please open a ticket in our [Support Portal](https://support.vilocify.com/hc/en-us/requests/new?ticket_form_id=44731177650067&tf_45678456155027=vilocify_component_update) so we may find a way to access this information. Failing to do so will result in Vilocify monitoring only publicly available information for that component.

## Why is the CVSS base score for a CVE in a notification different from the [National Vulnerability Database](https://nvd.nist.gov/)?

Vulnerabilities of notifications are scored in the context of the components that are assigned to a notification. Therefore, we preferably rely on the analysis of the actual vendor of a component instead of NVD. And a vendor sometimes scores the vulnerability differently than the original upstream CVSS of NVD (for example, a distribution package might use non-default compiler flags or non-default configuration of a component).

## When will you provide CVSS version 4.0?

The Forum of Incident Response and Security Teams (FIRST) has officially [announced CVSS version 4.0](https://www.first.org/cvss/v4.0/specification-document), the next generation of the Common Vulnerability Scoring System standard.

As of October 2024, Vilocify supports CVSS version 4.0. Vulnerability notifications in Vilocify will display CVSS version 4.0 for its vulnerabilities through both our [Portal](https://portal.vilocify.com/) and service's [API](../apiv2.html).

Note

We anticipate an industry-wide adoption of the latest version of the CVSS standard. To facilitate a smooth transition, our service will undergo an **adaptation period until the end of October 2025**. During this time, our notifications will include both CVSS version 3.1 and CVSS version 4.0 vectors fields. After the adaptation period, we will provide the CVSS version published by vendors.

## Why do you not share the impact of a notification?

With the introduction of CVSS 4.0, our methodology for communicating vulnerability impact has evolved.

Previously, under CVSS 3.1, we derived and shared specific "impact" statements (for example, "Exposure of Sensitive information") based on direct interpretations of vector metrics, for example as the Confidentiality not set equal to None.

CVSSv4 makes the distinction between "vulnerable" and "subsequent" system impacts (in favor of the CVSSv3 Scope: Changed/Unchanged). Consequently, we no longer provide generalized "impact" categories as previously shared. The understanding of a vulnerability's potential effects is now captured and communicated through the CVSS 4.0 metrics themselves.

## The vendor doesn't provide a CVSS version 3.¼.0. However, Vilocify notification includes both. How's the conversion done?

During the adaptation period between CVSS version 3.1 and CVSS version 4.0 standards, Vilocify will provide CVSS version 3.1 scores even when vendors only supply CVSS version 4.0. In such cases, Vilocify analysts evaluate the vulnerabilities and adapt the CVSS version 4.0 metrics to generate corresponding CVSS version 3.1 scores. This approach ensures consistency in the assessment and prioritization of vulnerabilities and security notifications by maintaining a common scoring system.

## What's the difference between the packages available for purchase for Vilocify for customers outside of Siemens?

If you are a Siemens colleague, reach out to us for an agreement for vulnerability monitoring of your projects or product development. For our external customers, see the MLFB structure in the [Service Description](../service-description/consumption-reporting.html#available-mlfbs).

## Who can I contact if I've additional questions not answered here?

- Patch Installation issues: Contact your vendor support
- For questions related to Vilocify, please open a [support ticket](https://support.vilocify.com/hc/en-us).
- If you are unsure whom to contact, feel free to contact us. We will forward your inquiry to the appropriate address if necessary.

# Frequently asked questions about Vilocify emails

## Why am I receiving emails from Vilocify?

There are a number of reasons for which our service sends emails to our customers, e.g.:

- When a monitoring list is created
- When users or components assigned to a monitoring list are modified
- When Vilocify publishes new notifications affecting components in your monitoring lists. **Please be aware that even if you never actively created a monitoring list within Vilocify Portal yourself, you still might be receiving mails from us.** This might be the case when somebody else created a monitoring list adding you as a subscriber.

## How can I unsubscribe from Vilocify notification emails?

You can modify the subscriber settings in the monitoring list tab of the Vilocify Portal. See [Managing Monitoring Lists#ModifySubscribers](../getting-started/managing-monitoring-lists.html#modify-subscribers) for more details. If you are only a reader of the monitoring list, please ask an owner of the list to modify the settings for you.

## How do I ensure that emails from Vilocify aren't marked as spam?

While we aren't aware of any instances where emails sent from our systems are marked as spam, we do recommend explicitly whitelisting our address [svm.ct@siemens.com](mailto:svm.ct@siemens.com) in your email client. In Outlook, right-click on an email sent from our system, go to **Junk**, then select the option **Never Block Sender** (see screenshot).

## Why do I not receive emails for notifications of all transitive child components?

The notifications for monitoring lists are only sent by email for the list itself and one level of child lists - that's, notifications for "grandchildren" lists aren't sent through email. This limits the load of emails, so the receivers can still process them and the service is still able to send these emails. In the portal you can show all transitive notifications with filtering by "Affecting monitoring lists."

# Getting started

## Access to Vilocify

To order a trial or full access to Vilocify, visit the [landing page](https://www.siemens.com/global/en/products/services/digital-enterprise-services/industrial-security-services/vulnerability-management.html) and fill the contact form.

## Guides

Here you find a couple of quick guides that demonstrate and explain common workflows in Vilocify.

- [How to create a monitoring list in order to be alerted when new notifications are published for specific components.](creating-a-monitoring-list.html)
- [How to request a new component, when it doesn't exist in the Vilocify database.](requesting-a-new-component.html)
- [How to properly understand the various fields and attributes of a notification.](understanding-a-security-notification.html)
- [How to edit existing monitoring lists](managing-monitoring-lists.html)

Info

If you encounter unknown terminology in our guides, you'll likely find definitions and answers

- by using the search on top of the page
- by consulting the [FAQs](../faq/index.html)
- by checking the [service description](../service-description/index.html), which explains key concepts of Vilocify.

# Creating a monitoring list

With a huge amount of monitored components and thousands of new notifications each year, not all the information within the Vilocify Portal is of interest to every single user.

Monitoring lists allow to filter only relevant information by allowing Vilocify users to select specific components they want to actually monitor. Typically, a monitoring list represents a specific product the user is responsible for (containing all the libraries, run-times, frameworks, etc. used for that product) or a specific infrastructure asset (containing components representing hardware, operating system, installed applications, etc.). Whenever new relevant information is published, subscribers of the list will be alerted via a notifications email, that's, a user-specific mail sent once daily, with all new notifications affecting a user's monitoring list.

## Start the monitoring list creation

To start the creation of a monitoring list, visit the [monitoring list tab of the Vilocify Portal](https://portal.vilocify.com/monitoringlists) (you might be asked to log in first) and then click on the "Create Monitoring List" button in the upper left corner (see screenshot below).

## Step 1: Edit basic data

In the first step of the creation process you need to provide some basic information about your monitoring list:

1. **Name** (mandatory): this is the name that will be displayed in the monitoring list table within the Vilocify Portal. Monitoring list names don't need to be unique, but we encourage you to give them descriptive names, for example, reflecting the name of the product or IT asset the list represents.
1. **Comment** (optional): you can add additional comments to a monitoring list, which will be shown to its subscribers.

Once you have filled the mandatory and optional elements as desired, click on the "Save" button on the bottom right of the window.

## Step 2: Select components

You are now able to select all the components you wish to monitor, for example, reflecting all used third-party components within a product you are responsible for, software running on an application server you manage, or IT assets within your own infrastructure. Whenever a new notification is published for a monitored component, you will be alerted via notification email.

To add or modify components, click on the Modify Components button as shown in the Screenshot. It will open the Select components overview in which you can edit your monitored components.

- Search for the desired component via the search field on the top left
- Check the box to the left of the component to select it
- If you can't find the desired component, try using different search terms. If you still can't find it, click on the "*Request Component"* under the search area, to place a [component request](requesting-a-new-component.html). Please note that each component request has to be analyzed by the Vilocify team, as such the requested component won't be immediately available to be added to your list. Once your component request is processed, you will be informed via email, and you can then add it to your monitoring list.

Once you have added (or requested) all the desired components to your list, click on the "Save" button on the bottom right of the window.

## Step 3: Define subscribers

In the next tab you can modify the subscribers of the monitoring list. Subscribers will be able to see the monitoring list within the Vilocify Portal (including its components and subscribers) and will receive notifications emails whenever a new notification is published for any component assigned to the monitoring list. The creator of the monitoring list is automatically added as a subscriber, further subscribers can be defined by clicking the "*Add subscriber"* button on the top left. This opens a further window, in which you can search by name or email address.

In the Add Subscriber popup, search for a user in your organization and configure two settings for them:

1. **Role**: Subscribers with the "Owner" role are able to modify the monitoring list, that's, they can change the basic data of a monitoring list (see step 2), the assigned components and its subscribers. Additionally, they're allowed to delete the list and add dependencies to other lists they can see. Subscribers with the "Reader" role have read-only access to the monitoring list.
1. **Priorities**: Vilocify notifications are always assigned a priority (see [corresponding page](understanding-a-security-notification.html) for additional information). If you wish your notifications email to contain only notifications of certain priorities, you can change this setting accordingly. Please note that changing this setting might lead to missing out on relevant security vulnerabilities.

Click the "Save" button on the bottom right of the popup window to add the new subscriber to the monitoring list.

## General remarks

- The [monitoring list tab of the Vilocify Portal](https://portal.vilocify.com/monitoringlists) shows only monitoring lists you are assigned to, they're not made public. If you wish to see a monitoring list created by another user, you need to be added as a subscriber of that list. Also, organization admins can see all lists in their organization.
- Anything defined within the creation process can be changed after the creation of a monitoring list by any of its subscribers with the role "Owner." See [corresponding Wiki page](managing-monitoring-lists.html) for additional information.
- Subscribers of a list will be notified of its creation or modification via email, only if you check the "Notify subscribers" checkbox before saving.

# Managing monitoring lists

Existing monitoring lists aren't set in stone, they can be modified, copied, or deleted. You can find corresponding instructions in the following paragraphs.

## Modifying monitoring lists

All the information provided during the monitoring list creation process can be modified at any time by any of the list's owners (or tenant admins). This includes the *basic data*, the *monitored components*, *dependencies*, and *subscribers*.

### Modify basic data

You can change the basic data of any monitoring list you are the owner of. This includes its name and the comment. For additional information regarding the single properties, please refer to [the corresponding section of the monitoring list creation guide](creating-a-monitoring-list.html#step-1-edit-basic-data).

In order to change a list's basic data, select the desired monitoring list in the [corresponding section of the Vilocify Portal](https://portal.vilocify.com/monitoringlists), select the "*Basic Data*" tab (see screenshot) and update the name and the comment of the list. Once you are done, click on "*Save*" to update your monitoring list. Please note that only subscribers with the role "*Owner*" can modify the basic data of a list.

### Modify components

Components of a monitoring list can be changed at any time. This is useful in instances when you want to add a new one (for example, because a new asset has been added within your infrastructure) or replace one (for example, because you updated a library and would therefore monitor the new version). To change the components, select the desired monitoring list in the [corresponding section of the Vilocify Portal](https://portal.vilocify.com/monitoringlists), select the "*Components*" tab (see screenshot on the bottom left) and click on the "*Modify components*" button on the top right.

This will open a new window (see screenshot) where you can add components by searching and selecting them from the left grid, or removing existing ones from the right grid. Please note that only subscribers with the role "*Owner*" can modify the components of a list. For additional information please refer to [the corresponding section of the monitoring list creation guide](creating-a-monitoring-list.html#step-2-select-components).

If you want to remove added components, click on "Only show selected Components," then deselect all unwanted components, and afterward save the updated list with a click on "Save."

### Modify subscribers

Subscribers of a monitoring list can be modified after its creation as well. New subscribers can be added, existing ones can be removed, their roles and notifications email priorities can be changed. To change a list's subscribers, select the desired monitoring list in the [corresponding section of the Vilocify Portal](https://portal.vilocify.com/monitoringlists), and then select the "*Subscribers*" tab (see screenshot on the bottom). From here you can do the following:

- Add a new subscriber by clicking on the "*Add subscriber*" button on the top and then search for the desired users.
- Changing an existing subscriber's settings by clicking on the pencil icon at the right of the corresponding row. From there you can:
  - change a subscriber's role by selecting the corresponding option.
  - change a subscriber's priorities by selecting the desired priorities.
  - remove an existing subscriber by clicking on the "*Remove*" button.
- To delete a subscriber, click the trash can icon at the right of the row.

Please note that only subscribers with the role "*Owner*" can modify the subscribers of a list. For additional information please refer to [the corresponding section of the monitoring list creation guide](creating-a-monitoring-list.html#step-3-define-subscribers).

### Modify dependencies

Owners of a list can create dependencies between monitoring lists within their organization by defining parent-child relationships between them. When such a parent-child dependency is set, any new notification affecting the child monitoring list will be included in the notifications email of subscribers of the parent monitoring list.

Warning

Anyone who can see the parent list, can see all child and descendant lists, even if they're not subscribed directly on the child list.

## Monitoring list actions

It's furthermore possible to perform certain actions on monitoring lists.

### Copy

This action allows to quickly create a new monitoring list, using the selected one as a reference to start from. Click on the copy icon (marked in the screenshot below) in the row of the monitoring list of which a duplicate should be created.

It will create a new monitoring list entry in the table, recognizable at the same name with "(copy)" added. To modify the new list, please follow the same instructions as an existing monitoring list would be modified.

### Pause

By pausing a monitoring list, the list will still be available within the Vilocify Portal, but it won't be included in the notifications email of subscribers assigned to it. To pause a list, disable the "Notification emails are sent for this list" section in the Basic Data overview, as marked in the screenshot at the bottom. Deactivated monitoring list are marked by a pause label in the table of all visible monitoring lists.

### Delete

Clicking the red "Delete" button in the Basic Data overview will delete the monitoring list. Please be aware that the action affects all assigned subscribers and is irreversible.

# Requesting a new component

With a huge amount of components in its database, chances are that most components you wish to add to your monitoring lists already exist within Vilocify. It can nonetheless happen that you aren't able to find the desired component, for example, because it's a rather obscure piece of software or a newly released version.

In such cases you are free to request new components through our request form. Once you submit your request, the Vilocify team will analyze it in order to ensure that a correct and reliable vulnerability monitoring is possible. This usually takes a few business days. Once your request is successfully processed, you will be alerted by email and can finally add the new component to your monitoring list. In case the Vilocify team wasn't able to successfully process your request, you will be alerted by email with clear instructions on how to proceed.

To ensure that the component request process runs smoothly, be aware of the following guidelines.

## High level approach

The goal of our component request process is to add components to the Vilocify database in such a way, that they reflect the naming conventions used in official vendor advisories. This guarantees that we can reliably match relevant security information from the many sources in our monitoring, to the affected components. Failing to follow these conventions could lead to potentially missing out on relevant information.

## Components suitable for security vulnerability monitoring

Not all components are suitable for security monitoring. It's therefore important to understand which types of components can be properly monitored.

### Valid components

- **Valid are in general all components which are publicly visible**. That means, they can be found or identified clearly by using an internet search engine such as Google. Only if a component is externally visible security vulnerability information can be expected from the various information sources. *Examples:*
  - `Microsoft Windows 10 Version 22H2`
  - `Red Hat Enterprise Linux Server 8`
  - `Mozilla Firefox 109.0`
  - `GNU glibc 2.37`
  - `Cisco IOS Software 15`
  - `ISC Bind 9.18.11`
  - `DENX Software Engineering u-boot v2023.04-rc1` (see remark about beta versions further below)
- **Hardware elements with the exact hardware model as well as the firmware / software version running on that hardware.** *Examples:*
  - `IOS XE on Catalyst 9200 Series Switches (All Versions)`
  - `SIMATIC S7-1510SP-1 PN CPU`
- **Drivers for a hardware component** need to be explicitly defined. They won't be automatically assigned to an operating system. *Examples:*
  - `NVIDIA NVS300 Graphics Driver 191.87`
  - `Alcor Micro USB Smart Card Reader Driver (All Versions)`
- **Optional plug-ins, modules, add-ons and extensions for a framework, browser or software application** need to be explicitly registered. *Examples:*
  - `NuGet Package Newtonsoft.Json 13.0.2`
  - `AdBlock Firefox AdBlock add-on 5.3.3`
  - `Benjamin Trott Perl Module Crypt::DES_EDE3 0.01`
- **Packages from operating systems**. While Vilocify recommends monitoring operating systems as a whole, it's also possible to request specific packages of Linux distributions. Packages can only be monitored for all versions but not version specific. Packages can also be monitored for specific versions of distributions. *Examples:*
  - `RHEL Package: telnet All Versions`
  - `Debian Package: apt All Versions`
  - `Debian 11 Package: bind9 All Versions`

### Invalid components

- **Basically all components which aren't externally visible**, that's can't be found or identified clearly by using an internet search engine. If a component isn't externally visible security vulnerability information can't be expected from the information sources.
- **Code-snippets** due to lack of security advisories from the vendor.
- **Individual file names** or **sub-components** which are installed on the system aren't valid as new components for monitoring. For example `.jar`, `.dll`, `.exe`, `.bin`, `.ocx`. This is by far the most frequent reason for declined component request. Please make sure the desired components are mentioned as such on an official vendor page. *Examples*:
  - `Microsoft Office DLL 6.0.81` (correct: `Microsoft Office 2007`)
  - `commons-logging-1.1.1.jar` (correct: `Apache Commons Logging 1.1.1`)
  - `sc.exe` (correct: `Microsoft Windows XP`)
- **A product family group needs to be split** into individual components for monitoring. for example the product family `Siemens SIMATIC Net` needs to be split into the individual component or modules which are deployed or required for monitoring that's `Industrial Ethernet SOFTNET-S7`, `SNMP OPC Server Basic`, `PROFINET SOFTNET PN IO`, `PROFIBUS S7-5613` and not the product family name `Siemens SIMATIC Net`.

## Create a component request

The form for requesting a new component can be opened from multiple points within the Vilocify Portal, for example by heading to the [components tab of the Vilocify Portal](https://portal.vilocify.com/components) and clicking on the *"Request Component"* button. Component requests sent via email won't be processed. There are multiple fields you need to fill in before being able to submit the request.

**Vendor:** The vendor is the company, organization, open source community, or author of the desired component. Some vendors are obvious, for example Microsoft, Oracle, Red Hat, Adobe, Apache, but sometimes it isn't clear how to define the vendor correctly. The following guidelines provide some guidance for such cases:

- If the component wasn't developed by a company, organization or open source community, use the full author name as the vendor, for example *Matt Johnston* for the component *Dropbear*.
- If the company, organization, open source community or author is unknown, use the website hosting the component, for example *[gmplib.org](http://gmplib.org/)* for *The GNU Multiple Precision \*\*Arithmetic Library (GMP)*.
- GitHub, SourceForge, Maven, npm etc. are almost never valid vendors, as they simply are code repositories.
- Use official vendor pages to determine the correct vendor name.

**Component Name:** The component name is the name used to describe the software / hardware component as given by the vendor. The component name isn't the name of an installation file, download file, binary file or sub-component, that isn't usually .jar, .dll, .exe, .msi files. The following rules are defined to name the "component name" field consistently for all types of components:

- Use the name exactly as used by the vendor to describe the component, for example in official product page or vendor advisories. Avoid component names not coming from official vendor pages.
- The full name should be used but an abbreviation can be added in brackets at the end of the component name. for example `SUN Java Architecture for XML Bindings (JAXB)`.
- There is no need to include the vendor name in the "component name" field usually, only the component name is required for example *Windows*, not *Microsoft Windows*.
- Some components are written in different programming languages. In such instance, please specify the desired variety explicitly. Example: *Xerces XML Parser (for Java)* vs. *Xerces XML Parser (for C++).*

**Version:** The version field is for the version of the component. Use the following guidelines for the version field:

- The requested version must exist, and it should be possible to find hints of its existence on the vendor's website.
- The granularity of the requested version should match the granularity of the versions mentioned in official vendor's advisories. For example, it doesn't make sense to register *Microsoft Internet Explorer version 11.0.9600.18762* because Microsoft doesn't publish security bulletins on that level of granularity. Instead, *Microsoft Internet Explorer 11* should be used.
- It's possible to request a wildcard version to monitor whole branches of a component. This is especially useful for components that receive frequent security updates that change its version. Instead of requesting for example *Tomcat 8.0.31*, *Tomcat 8.0.32*, *Tomcat 8.0.33*, etc. every few weeks, you can simply request *Tomcat 8.0.x* or even *Tomcat 8.x*.
- Some components can be installed on various operating system platforms. Please specify the required platform in the version section for example *IBM WebSphere Application Server 7.0.0.21 for Windows*, *Adobe Acrobat Reader for Apple macOS*.
- Don't enter multiple versions in the field for one component. A new component for each individual version needs to be created for example the following entry with multiple versions is incorrect: *Microsoft Windows 7, 8, 2010 Server*. Three new component requests are required: *Microsoft Windows 7*, *Microsoft Windows 8*, *Microsoft Windows 2010 Server.*
- Don't use letters like "v" or "version" in the version field if the vendor of the component doesn't explicitly also use this naming convention. Also don't add additional comments or information which the vendor doesn't use in the version part for example self-compiled bug fixes.
- Relational characters such as `>`, `<`, `>=`, `<=` aren't allowed as this may result in an unclear component assignment.
- It's possible to request beta versions, release candidates, and so on, in the same manner as any other stable version. However, please note that often times, vendors don't disclose vulnerabilities affecting such versions. As such, using these types of versions might result in a loss of information that's made available for stable versions.

**URL:** Link to the vendor's page where information about the component can be found. Use the following guidelines for the URL field:

- URL should point to a page where it becomes clear that the requested component (ideally including its version) really exists. If you can't find an official page for the desired component, it might be an indication that the component request doesn't make sense.
- It's usually not sufficient to just add a vendor's top level domain, for example  should be used instead of just  for the component *SUSE Linux Enterprise Server (SLES)*.
- Avoid using non-official vendor links, for example re-seller sites, Maven or npm links, and so on.

**Security URL:** Link to a web page where component specific security information like security advisories or security bulletins can be found, for example  for SUSE products.

Some vendors provide security information only to customers with active support accounts, without making it available to the public. If you request a component (or add one to your monitoring list) where this is the case, please contact us via our [Support Portal](https://support.vilocify.com/hc/en-us/requests/new?ticket_form_id=44731177650067&tf_45678456155027=vilocify_component_update) so we may find a way to access this information. Failing to do so will result in Vilocify monitoring only publicly available information for that component.

**Comment:** An optional free text field that allows users requesting a new component to add any information that might facilitate the correct processing of the request.

## Submitting a component request

Once you have filled in all the mandatory fields you can submit the request form by clicking on "Submit." The requested component won't be immediately visible within the Vilocify Portal. Instead, the Vilocify team will analyze your request, determining whether a reliable vulnerability monitoring is possible with the provided information. Once the Vilocify team processed your request, you will be informed by email. If the component request was successfully processed, you will be able to find the corresponding component within our Vilocify Portal, ready to be added to your monitoring lists as needed. In cases where it wasn't possible to add the desired components you will be informed as well, including information about why it wasn't possible to successfully process your request. You are then free to use the link in the notification email to providing additional information / clarification.

The date on which a component has been added to the Vilocify service is marked in the "Monitored Since" field visible within the Vilocify Portal and the Vilocify REST API. Vulnerabilities published before the "Monitored Since" date of a component might not be assigned to a component, that's the Vilocify service doesn't currently offer retrospective monitoring. You can search for an older version or an "All versions" of that component within Vilocify. If all this doesn't help, you need to search the Internet.

# Understanding a security notification

Monitoring List subscribers receive **notifications emails** when new or updated relevant **security notifications** are published. A notifications email contains a summary of all notifications relevant for its receiver, grouped by affected monitoring lists. Each notification summary contains a link to the full version of the security notification available on the Vilocify Portal. In order to properly assess the possible impact of a security notification, it's vital to understand all information contained within. The following paragraphs describe the various parts of a typical security notification.

## Title

The title of a security notification provides important information at a glance. Titles generally consists of three parts:

- **Affected Systems**: A summary of affected products and versions.
- **Short Description**: A summary of the vulnerability described within the notification
- **Identifiers**: Based on availability either an advisory or patch identifier, a vulnerability identifier or fixed versions

**Example**:

## Overview

The overview is the first tab of a Vilocify notification, and it's purpose is to convey the most important attributes of a notification in a structured manner. It consists of several elements:

- **Priority**: A rating that describes the severity of the information contained within a notification. Factors like for example consequences of a successful exploitation, likelihood of exploitation and attack vector are considered when assigning a notification priority. When available, a notification priority is based on the priority provided by the vendor, but the Vilocify team reserves the right to alter this according to their own analysis. There are five possible ratings:
  - **Critical**: Typically used for remote attacks with critical impact which cause full compromise of the system. Investigate immediately and follow recommended action as soon as possible with urgency.
  - **Major**: Typically used for remote or local attacks with significant impact but no full compromise of system. Investigate as soon as possible and follow recommended action.
  - **Minor**: Typically used for local attacks with minor impact to system. Follow the recommended action in a reasonable amount of time.
  - **Information Only**: Typically used for notifications that aren't directly related to vulnerabilities but could have a security impact nonetheless, for example updates of trusted root certificate authorities or time zone information.
  - **Withdrawn**: The vulnerability has been withdrawn by the vendor. It's no longer valid and can be disregarded.
- **Action:** Recommended action necessary to mitigate the issues described within the notification.
- **Attack Vector:** Describes where an attacker initiates an attack to exploit the vulnerabilities described in the notification.
- **Vendor Affected components:** A list of components that the vendor explicitly mentions as affected. It differs from the components in the Components tab as this list is provided directly by the vendor. Not all components mentioned in this section necessarily exist within the Vilocify Portal, as components in the Vilocify database are added only when explicitly requested through a component request.
- **Solution:** Indicates what type of solution is available. This value is derived from the corresponding CVSS metric (Remediation Level). It contains details about how to mitigate the vulnerabilities described in the security notification and might suggest to update to fixed versions, to apply patches provided by the vendor, to implement certain workarounds, etc. Vilocify follows a patch oriented approach, providing actionable information for a collection of vulnerabilities in one notification and would therefore also collect several vendor advisories to one notification, if they share the same solution.
- **Description:** Contains a detailed description of the vulnerabilities included in this notification. These descriptions are generally taken from the official vendor advisories, when available.

**Example**:

## Components

The next tab, "Components" contains a list of all Vilocify components assigned to this notification. If a user has a monitoring list containing one of the components in this section, this notification is relevant for him and will be included in his notifications email.

**Example**:

## Vulnerabilities

In the next tab "Vulnerabilities" as shown in the screenshot below, are all vulnerabilities listed for a certain notification. A vulnerability often contains a [CVEs (Common Vulnerabilities and Exposures)](https://cve.mitre.org/), which are identifiers assigned to specific vulnerabilities by so-called [CVE Numbering Authorities](https://cve.mitre.org/cve/request_id.html). Vilocify publishes notifications for every CVE assigned by the Mitre Corporation, thus achieving CVE completeness. In addition, we also monitor the European Vulnerability Database (EUVD) and complement thereby CVE completeness with notifying vulnerabilities, not assigned a CVE, thus achieving EUVD completeness as well. Not all vulnerabilities have assigned CVEs. Unlike the [National Vulnerability Database](https://nvd.nist.gov/) by the NIST and many security information providers, Vilocify publishes notifications also for vulnerabilities without CVEs.

**Example:**

### CVSS score

The [Common Vulnerability Scoring System (CVSS)](https://www.first.org/cvss/) is a standard that allows to capture the most important metrics of a vulnerability in a structured manner, allowing to calculate a score reflecting its severity. Each Vilocify security notification has at least one CVSS vector for each reported vulnerability. Vilocify notifications support both [CVSSv3.1](https://www.first.org/cvss/v3.1/specification-document) and [CVSSv4.0](https://www.first.org/cvss/v4.0/specification-document) standards. The dual availability of CVSSv3.1 and CVSSv4.0 standards is part of a transition period from the end of October 2024 to the end of October 2025. During that transition period, Vilocify provides a CVSSv3.1 vector for every vulnerability that's scored with CVSSv4. After the transition period Vilocify only reports the CVSS version provided by the vendor, or CVSSv4 in the case where the vendor provides 3.1 and 4.

Note

Notifications published before 2017 are generally scored according to the CVSSv2 standard. Starting from October 2024, Vilocify will provide CVSSv4.0 scores whenever available.

The CVSS vectors of Vilocify vulnerabilities are determined in one of two ways:

- If the vendor provides its own CVSS scores in its vendor advisory or other official sources, Vilocify notifications report the provided scores.
- In cases where the vendor doesn't provide any CVSS score, Vilocify will independently score the vulnerability(ies).

It's important to note that Vilocify doesn't rely on the scores provided by the NVD. Please see the [FAQ](https://docs.vilocify.com/faq.html#why-is-the-cvss-base-score-for-a-cve-in-a-notification-different-from-the-national-vulnerability-database) for further information.

Vilocify doesn't provide the [environmental metrics](https://www.first.org/cvss/specification-document#Environmental-Metrics) of a vulnerability, as it has no knowledge of all the differing environments a component is used in. Determining the correct environmental metrics (if needed) is the responsibility of each product manager or IT asset responsible.

There isn't necessarily a correlation between notification priority and CVSS Base Score. The reason for this is that there are cases where CVSS doesn't address all necessary characteristics having an impact on the priority.

**Example:**

Extending a row in the vulnerability overview of a notification opens following visualization that explains the CVSS vector.

## Related links

### Vendor advisory

A list of links to official advisories provided by the vendor. They often contain additional information.

### References

A list of further resources that might contain additional information relevant to the vulnerabilities described in the notification.

### CVEs

This section lists all CVEs assigned to the vulnerabilities within the security notification, if available.

**Example:**

## Update history

The information contained within a notification can be subject to change over time, for example the vendor might correct some mistakes made in previous versions of the advisory, security fixes could be provided only a certain time after the vulnerability is made public, etc. Vilocify security notifications are updated accordingly and changes are documented in this section. In case of important updates, a security notification is included again in the notifications email of affected users.

The following scenarios are considered as an important update:

- A new solution is available.
- The existing solution is changed due to a new patch, a new version or other similar reason.
- The priority of a notification increases due to for example a vulnerability being actively exploited.
- Official reports (from trusted sources) that a vulnerability is being actively exploited become available and thus the Exploit Code Maturity changes (for example FBI or CISA reports).
- New Vilocify affected component is added to the notification.
- The end of life date is rescheduled to an earlier point in time.
- New security impact is introduced when a vulnerability is added to a notification. For example a notification initially affects only Confidentiality and affects Availability with the addition of the new vulnerability.

The following scenarios **aren't** considered as an important update:

- Updating description without new critical information.
- Adding new CVE without affecting the overall criticality and solution of the notification.
- Fixing spelling mistakes.
- Adding new reference links.
- Adding Nessus Plugins.

**Example:**

## End-of-life notifications

Whenever possible, the Vilocify service generates [End-of-life](https://portal.vilocify.com/notifications#keywords=eol) notifications 18 months before a vendor stops supporting a specific component. In instances where such information is made available less than 18 months in advance, the time period can change accordingly. End-of-life notifications are very similar to regular notifications, but lack some attributes like CVSS metrics, CVEs, Attack Vector, etc.

EOL information is provided on a "best effort" basis. This means publishing vulnerability notifications has priority and there is no guarantee that a EOL information can always be provided. The Vilocify service creates EOL notifications as far as they're known to it.

The Vilocify service monitors thousands of security sources for vulnerability information affecting **components** in its database. Vilocify users can use the [Vilocify Portal](https://portal.vilocify.com) to define **monitoring lists** in order to proactively alert its **subscribers** whenever new **notifications** affecting these components are published. In order to use the Vilocify service optimally, it's important to understand its base concepts and workflows, described in this service description.

# Component requests

Components are added to the Vilocify database only when they're explicitly requested by our users. If you can't find the component you need to add to your monitoring list, please request it through our [request form](../getting-started/requesting-a-new-component.html) or the [REST API](../apiv2.html). The component request is then processed by Vilocify's analysts to ensure the component in the requested version actually exists. If our analysts can't identify the component, we reject the component request. The more information you provide in your component request the more likely it's that we don't reject it.

# Component

In the context of the Vilocify service, a component can be any piece of software (both [open source](glossary.html) and [commercial off-the-shelf](glossary.html)), hardware, or combination thereof. Components can be e.g.

- operating systems like `Microsoft Windows 10` or `Red Hat Enterprise Linux Server 7`
- software libraries like `OpenSSL 1.0.2n` or `GNU Gzip 1.9`
- specific hardware like `Fujitsu Primergy RX300 S2 Server` or `Cisco Catalyst 3750 Series Switches`
- combinations of hardware and software (when they're provided by the same vendor) like `Cisco IOS on Catalyst 2960-XR Series Switches 15.x` or `Juniper Junos OS on SRX5800 Platform`
- cloud components, for which the vendor is releasing security-related information like for example `Amazon AWS CloudFront`

The full list of components currently monitored by the Vilocify service can be found in the [components section of the Vilocify Portal](https://portal.vilocify.com/components), additionally we also offer a [REST API](../apiv2.html). Components in the Vilocify database have (amongst others) the following attributes:

- **Vendor**: The company, organization, open-source community, or author of the desired component.
- **Component Name**: The name used to describe the software/hardware component as given by the vendor.
- **Version**: The specific version of the software/hardware. Additionally, the Vilocify service monitors *wildcard* components, which group a specific range of versions. For example the wildcard version *2.x* monitors all versions between version 2.0 (including) and version 3.0 (not including). Furthermore, *All Versions* components represent every subsequent version from the time the *All Versions* component is added to the Vilocify monitoring database.
- **URL** Link to the official vendor's page for the component.
- **Monitored Since**: The date since when the component has been added to the Vilocify monitoring service. This means that any vulnerabilities disclosed before that date might not be assigned to that component.
- **EOL**: End of Life, stating whether support for a component has been discontinued by its vendor. Generally no security updates can be expected for components marked as EOL. Not all vendors provide EOL information for their products. Thus, components without an EOL date aren't necessarily still supported. Vilocify provides support information on a best-effort basis. Components are marked as EOL only when an official vendor statement exists, meaning that the support status isn't automatically derived from for example the age of a component or the existence of newer versions.

Components in the Vilocify database are subject to regular reviews and quality checks. While small corrections of single attributes might occur, a registered component will never be updated in such a way that it represents a semantically different one. In case logical duplicate components are detected, one will be deactivated and replaced by the other. All resources (monitoring lists, existing notifications, API responses, etc.) will be automatically updated accordingly.

# Consumption reporting

Vilocify customers buy different sized packages of API requests and component requests (see [MLFBs](#available-mlfbs)) that are consumed over a time period. We call this time period the *consumption period*. The ordered packages can't be used after the consumption period ends and unused API requests can't be transferred to the next consumption period. Consumption periods of an [organization](membership-and-organization.html) can't overlap.

During the consumption period Vilocify tracks consumption based on the API requests that organizations and their users make. Admins can see all usage of all members of their organization in the [Portal](https://portal.vilocify.com/consumption-report/total) At the end of a consumption period every admin receives an email with a report detailing the usage of the period. The report includes the overall amount of ordered packages and usage details of every membership. Users can see their own consumption and the overall organization consumption on the dashboard, but can't see usage of other users.

When the organization runs out of available requests no user of the organization can perform API requests. This includes usage of the [Web Portal](https://portal.vilocify.com), because the portal uses the API. There is no enforcement to restrict a single user, or in other words:

Warning

A single user can consume all requests allocated to an organization

## What happens when you run out of requests

When an organization used up all the ordered requests, the API will respond with a `402` error code and almost all features in Web UI will be locked. It's possible to order extensions during a consumption period. Contact your regional Siemens sales person or open a [support ticket](http://support.vilocify.com/hc/en-us).

## API details

API Details regarding consumption reporting and enforcement are documented in the [API documentation](https://portal.vilocify.com/documentation#?route=overview--api-consumption).

## Available MLFBs

For our external customers, Vilocify can be procured using MLFB order numbers. These are listed for your reference in the following table.

| Name of the product    | MLFB                                                                                                  | Description                                                                                      |
| ---------------------- | ----------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------ |
| API V2 - Small         | [9LA1110-1DV14-1AA2](https://sieportal.siemens.com/en-ww/products-services/detail/9LA1110-1DV14-1AA2) | 36000 API requests for 3 memberships in one organization for one year                            |
| API V2 - Standard      | [9LA1110-1DV14-1AB2](https://sieportal.siemens.com/en-ww/products-services/detail/9LA1110-1DV14-1AB2) | 300000 API requests for 50 memberships in one organization for one year                          |
| API V2 - Individual\*  | [9LA1110-1DV14-1AC2](https://sieportal.siemens.com/en-ww/products-services/detail/9LA1110-1DV14-1AC2) | individual offer                                                                                 |
| API V2 - Extension     | [9LA1110-1DV14-1AD2](https://sieportal.siemens.com/en-ww/products-services/detail/9LA1110-1DV14-1AD2) | additional 6000 API requests for the current period (only in combination with Small or Standard) |
| API V2 - Initial Setup | [9LA1110-1DV14-2AB2](https://sieportal.siemens.com/en-ww/products-services/detail/9LA1110-1DV14-2AB2) | Setup organization and 8 hours of technical support                                              |
| API V2 - New component | [9LA1110-1DV14-3AA2](https://sieportal.siemens.com/en-ww/products-services/detail/9LA1110-1DV14-3AA2) | Mapping or creation of one new component submitted through a component request                   |

\* Please note that individual offers are available upon request.

# Glossary

## Definitions and abbreviations in the context of the Vilocify service

| Term                       | Description                                                                                                                                                                                                       |
| -------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Advisory                   | Vendor published statement on a validated vulnerability and instructions how to handle this issue.                                                                                                                |
| Component                  | See description of [component](component.html).                                                                                                                                                                   |
| CPE                        | [Common Platform Enumeration](https://nvd.nist.gov/products/cpe): structured naming scheme for IT components provided by NIST National Vulnerability Database (NVD)                                               |
| CVE                        | Common Vulnerabilities and Exposures: a unique, alphanumeric identifier assigned by the [CVE Program](https://www.cve.org/), referencing a specific vulnerability                                                 |
| CVE completeness           | See description of [notification](security-notification.html)                                                                                                                                                     |
| CVSS                       | [Common Vulnerability Scoring System](https://www.first.org/cvss/) is an open framework for communicating the characteristics and severity of vulnerabilities.                                                    |
| EOL                        | End of Life date, see description of [component](component.html).                                                                                                                                                 |
| EUVD completeness          | See description of [notification](security-notification.html)                                                                                                                                                     |
| MLFB                       | German Acronym for "Maschinenlesbare Fabrikatebezeichnung," which translates to "Machine-readable Product Indication," is a product code used by Siemens to identify a specific product or component.             |
| Monitoring List            | See description of [monitoring list](monitoring-list.html).                                                                                                                                                       |
| Monitoring List Owner      | Subscribers of a monitoring list who have the permission to modify the list. In particular, adding or removing subscribers, adding or removing components, editing its basic data, or deleting the list entirely. |
| Monitoring List Subscriber | User assigned to a monitoring list and will therefore receive notifications emails containing relevant notifications.                                                                                             |
| Notification               | See description of [notification](security-notification.html).                                                                                                                                                    |
| Notifications Email        | See description of [notifications email](notifications-email.html).                                                                                                                                               |
| NVD                        | [National Vulnerability Database](https://nvd.nist.gov/)                                                                                                                                                          |
| NIST                       | [National Institute of Standards and Technology](https://nist.gov)                                                                                                                                                |
| CISA                       | [Cybersecurity and Infrastructure Security Agency](https://cisa.gov) (America's Cyber Defense Agency)                                                                                                             |
| OSS                        | Open-source software: Software (components) from source code that's free available, but not always for free.                                                                                                      |
| Security Notification      | See description of [notification](security-notification.html).                                                                                                                                                    |
| Vilocify                   | Vulnerability Intelligence Service                                                                                                                                                                                |
| Vulnerability              | Weakness of an component (or product) that can be exploited by an attacker                                                                                                                                        |

# Membership and organization

Memberships and organizations are Vilocify's means of tenant separation on our SaaS offering. In most places memberships and organizations are hidden for normal users. However, they're important concepts for admin users and users of the REST API.

Vilocify organizes is customer in **organizations**, sometimes referred to as tenants. Once you ordered Vilocify, an organization is created in Vilocify and one person, specified during the ordering process, becomes the admin of the organization. The admins can invite users to their organization with a certain role and expiry date. One user can be in multiple organizations and one user can have multiple roles in one organization.

A **membership** is a user with a certain role in an organization. Vilocify internals often don't operate on a user directly, but on the membership (for example a subscriber of a monitoring list is actually not a user, but a membership). Memberships of one organization can't access data of other organizations. That's the reason why you need to select a membership during the login to Vilocify.

The amount of memberships is limited based on the package you (or your admin) ordered. See [Consumption Reporting](consumption-reporting.html) for details.

# Monitoring list

With a huge amount of monitored components and thousands of new notifications each year, not all the information within Vilocify is of interest to every single user. Monitoring lists are the tool with which users can filter only the relevant information, as well as set up alerting functionality via notifications email.

Monitoring list consists of various elements:

- **Basic Data**: Generic information about the list, for example its name, comments, etc.
- **Components**: A set of components a user wants to monitor, for example based on components within a product or IT asset under his responsibility.
- **Subscribers**: A set of users assigned to the list, who will then receive relevant notifications via email whenever new notifications are published for the monitored components.

Monitoring lists aren't public, that's a Vilocify user can only see lists he is subscribed to. Only admins can see all monitoring lists of their organization, regardless of whether they're subscribed to the list or not. Users can create as many monitoring lists as needed, for example to reflect IT assets under their responsibility. For additional guidance on how to create a monitoring list, please consult the [corresponding Vilocify Docs page](../getting-started/creating-a-monitoring-list.html). Furthermore, it's possible to define parent-child dependencies between monitoring lists. Notifications affecting a child monitoring list will be included in the notifications email of the subscribers of the parent monitoring list. You can find all monitoring lists you are assigned to either in the [monitoring list section of the Vilocify Portal](https://portal.vilocify.com/monitoringlists#ml=first), or the [Vilocify REST API](../apiv2.html).

# Notification email

Subscribers of monitoring lists are alerted of relevant notifications via a notifications email. This email is sent once daily and contain all the notifications relevant for the user, grouped by affected monitoring lists. On days without relevant notifications, users won't receive such an email.

# Roles and permissions

Vilocify has two permission concepts: one for general access to the application with roles *user* and *admin*, and one for access to monitoring lists (through subscriptions) with roles *owner* and *reader*. The following describes how the roles generally interact and what permissions they entail.

- The user role can
  - Create monitoring lists
  - See monitoring lists they're subscribed to (including all child dependencies, and recursively all descendants)
  - Modify monitoring lists if they're subscribed as *owner* to the list (including creating and modifying monitoring list dependencies to lists they can see)
  - Create and modify own component requests
  - See own component requests
  - See any notification and any component
  - See all memberships of their organization
  - Create API Tokens
- The user manager role can do everything the user role can, and additionally:
  - Invite users or user managers into the organization
  - Expire users from the organization
- The security officer role can do everything the user role can, and additionally:
  - See all monitoring lists in the organization, even if not subscribed on the list
  - Modify monitoring lists, even if not subscribed to the list
  - See and modify any component request of the organization, even if not the author of the request
- The admin role can do everything the user role can, and additionally:
  - See all monitoring lists in the organization, even if not subscribed on the list
  - Modify monitoring lists, even if not subscribed to the list
  - Invite and expire admins, security officers, user managers, and users into the organization.
  - See and modify any component request of the organization, even if not the author of the request

Note

API Tokens always have the same permissions as the membership who created it.

# Security notification

The Vilocify service constantly monitors thousands of sources (like official vendor advisory pages, vulnerability databases, security mailing lists, commercial security providers, security researcher blogs, etc.) for new vulnerability information. Whenever Vilocify detects vulnerability information affecting any of the components in its monitoring, a corresponding security notification is published. A security notification includes a detailed description of all vulnerabilities fixed in a new vendor patch, version or package for the affected components. We're publishing notifications for every CVE that's published at [NVD](https://nvd.nist.gov/vuln/search), thus offering CVE completeness from September 2020 of components that exist in our database. In addition, we also monitor the European Vulnerability Database (EUVD) and complement thereby CVE completeness with notifying vulnerabilities, not assigned a CVE, thus achieving EUVD completeness as well. More information regarding our notifications can be found in the [corresponding section of the Vilocify Docs](../getting-started/understanding-a-security-notification.html).

Most times security notifications are based on official vendor advisories. In cases where no official advisory is available, Vilocify analyzes and processes the available information in order to create a notification.

You can find all notifications published by the Vilocify service in the [notification section of the Vilocify Portal](https://portal.vilocify.com/notifications), and the [Vilocify REST API](../apiv2.html).
# Walkinside documentation

# Contact

For additional questions, please [contact us](https://www.siemens.com/global/en/products/automation/topic-areas/process-industries/software-for-process-industries/software-for-process-industries-contact.html)

- [Overview](overview.html)
- Walkinside API
  - [Overview](walkinside-api/index.html)
    - [Getting Started](walkinside-api/getting-started.html)
    - [Web Components](walkinside-api/web-component.html)
    - [Javascript API](walkinside-api/javascript-api.html)
    - [Project and User Management API](walkinside-api/project-management-api.html)
- [Contact](contact.html)

# Walkinside Overview

## Introduction

**Walkinside Services** offer a robust 3D visualization platform that transforms how enterprises interact with their digital twins. By enabling dynamic navigation through virtual asset models, Walkinside leverages immersive environments to enhance operational planning, training, and safety assessments.

*Experience high-fidelity 3D visualization for optimal asset management.*

## Elevate Your Application

Integrate our immersive 3D viewer into your applications to support diverse use cases—from operational walkthroughs and review sessions in processing plants to comprehensive familiarization and training based on accurate digital twins.

### Key Benefits

- **Accelerated Project Delivery**: Seamlessly integrate 3D visualization to expedite project execution, reduce commissioning times, and shorten the market delivery timeline through enhanced collaborative tools.
- **Cost Efficiency**: Maintain consistent access to up-to-date plant data within an easily navigable 3D viewer, reducing operational overhead and enabling cost savings.
- **Enhanced Safety and Sustainability**: Plan and execute plant operations with precision using geolocation-based asset models, optimizing for both safety and sustainability.

Thank you for choosing Siemens Walkinside. We are excited to see what you build with our APIs!

For more insights into how Walkinside can transform your operations, [contact our team](contact.html).

# Walkinside API Overview

## Introduction

Welcome to the official developer portal for Walkinside, your comprehensive guide to integrating and leveraging our powerful 3D visualization and RESTful service APIs. Whether you're looking to enhance your applications with immersive 3D environments or manage projects and user data effectively, our portal provides all the resources you need.

### Empower Your Workforce

Utilize our APIs to create custom solutions that offer your users hands-on, interactive experiences. Whether for training new operators or for conducting precise safety assessments, Walkinside serves as your gateway to a more informed and prepared team.

### Strategic Advantages

- **Rapid Integration**: Embed advanced 3D visualization directly into your solutions to minimize downtime and accelerate your workflows.
- **Operational Excellence**: With real-time access to a fully interactive 3D model, streamline decision-making processes and enhance operational efficiencies.
- **Informed Decisions**: Harness the power of geolocation-based data visualization to strategically manage and deploy critical assets, ensuring both safety and sustainability.

## Getting Started

Before diving into the details of our APIs, here are a few initial steps to get you set up and ready to go:

1. **Access Credentials**: Make sure you have the necessary credentials to access our APIs. [Contact us](../contact.html) if you need access keys.
1. **Environment Setup**: Set up your development environment to make API calls effectively. Refer installation manual.

## API Documentation

Our API suite is divided into two main categories, each tailored for specific functionalities and use cases:

### 1. [WalkinsideJS API](getting-started.html)

Dive into our comprehensive guide on using WalkinsideJS for integrating immersive 3D visualizations directly into your web applications. Ideal for training simulations, project reviews, and much more.

- [Adding WalkinsideJS to your application](getting-started.html#adding-walkinsidejs-to-your-application)
- [Getting Started with WalkinsideJS](getting-started.html#creating-a-view)
- [Loading a Project](getting-started.html#loading-a-project)
- [Project API and Examples](javascript-api.html#project)

### 2. [Project & User Management API](project-management-api.html)

Explore our RESTful services for managing project data, user configurations, and system interactions efficiently.

- [Swagger](project-management-api.html)

## Support and Community

Join our community or get support:

- [Issue Tracker](https://github.com/walkinside/sdk/issues)
- [Contact Support](../contact.html)

# WalkinsideJS Getting Started Guide

## Table of Contents

1. [Adding WalkinsideJS to your application](#adding-walkinsidejs-to-your-application)
1. [Creating a View](#creating-a-view)
1. [Loading a Project](#loading-a-project)

______________________________________________________________________

## 1. Adding WalkinsideJS to your application

To get started with WalkinsideJS, follow these three easy steps:

1. **Set up COMOS Walkinside Server**: Ensure you have a COMOS Walkinside Server, including the Rendering Service feature. This requires that the server has a GPU supported by COMOS Walkinside.

1. **Load WalkinsideJS Script**: Add the `walkinside.js` script to your HTML application:

   ```html
   

   ```

   (If you customized your installation, the `/walkinside-js/` path may be different.) 3. **Add a Container Element**: Include an HTML element in your page that will serve as a container for the 3D view. Any block element, like a `
      `, will work.

After the script is loaded, you can start using the global `walkinside` object. Its interface is described in the "API Reference" section below.

### Versioning

WalkinsideJS follows the semantic versioning scheme () to indicate to clients when there is compatibility breakage.

______________________________________________________________________

## 2. Creating a View

Global "walkinside" object

**`walkinside.createViewAsync(options) → Promise`**

Initializes a new 3D viewer within a specified container element and connects it to the Walkinside backend.

### Parameters

- `options` (Object):
  - `backendAddress` (string): The server URL where the Walkinside backend is hosted.
  - `targetElement` (HTMLElement): The DOM element that will host the 3D view.

### Example

```javascript
walkinside.createViewAsync({
    backendAddress: 'https://example.com/walkinside-backend',
    targetElement: document.getElementById('wi-viewer-container')
}).then(function(view) {
    console.info('View created successfully.');
}).catch(function(error) {
    console.error('Error creating view:', error);
});

```

______________________________________________________________________

## 3. Loading a Project

**`view.loadProjectAsync(options) → Promise`**

Loads a specified project into the initialized view.

### Parameters

- `options` (Object):
  - `projectAddress` (string): The URL of the Walkinside project.
  - `token` (string): Authentication token.

### Example

```javascript
view.loadProjectAsync({
    projectAddress: 'https://example.com/projects/123',
    token: 'your_access_token'
}).then(function(project) {
    console.info('Project loaded successfully:', project);
}).catch(function(error) {
    console.error('Error loading project:', error);
});

```

# walkinside-js

## Classes

### Coordinate

Represents a 3D coordinate.

#### Constructors

##### Constructor

> **new Coordinate**(`x`, `y`, `z`): [`Coordinate`](#coordinate)

###### Parameters

| Parameter | Type     |
| --------- | -------- |
| `x`       | `number` |
| `y`       | `number` |
| `z`       | `number` |

###### Returns

[`Coordinate`](#coordinate)

#### Properties

| Property | Modifier | Type     |
| -------- | -------- | -------- |
| `x`      | `public` | `number` |
| `y`      | `public` | `number` |
| `z`      | `public` | `number` |

______________________________________________________________________

### Point3

Represents a 3D point with x, y, z coordinates.

This class is JSON-compatible: instances can be created directly from JSON.parse and serialized with JSON.stringify.

#### Example

```typescript
const point = new Point3(1.0, -2.0, 4.65);
const json = JSON.stringify(point);        // '{"x":1,"y":-2,"z":4.65}'
const restored: Point3 = JSON.parse(json); // { x: 1, y: -2, z: 4.65 }

```

#### Constructors

##### Constructor

> **new Point3**(`x`, `y`, `z`): [`Point3`](#point3)

Creates a new [Point3](#point3) instance.

###### Parameters

| Parameter | Type     | Description       |
| --------- | -------- | ----------------- |
| `x`       | `number` | The X coordinate. |
| `y`       | `number` | The Y coordinate. |
| `z`       | `number` | The Z coordinate. |

###### Returns

[`Point3`](#point3)

#### Properties

| Property | Modifier | Type     | Description       |
| -------- | -------- | -------- | ----------------- |
| `x`      | `public` | `number` | The X coordinate. |
| `y`      | `public` | `number` | The Y coordinate. |
| `z`      | `public` | `number` | The Z coordinate. |

______________________________________________________________________

### Pose

Represents a pose with position and orientation.

#### Constructors

##### Constructor

> **new Pose**(`position`, `orientation`): [`Pose`](#pose)

Constructs a new Pose.

###### Parameters

| Parameter     | Type                        | Description                             |
| ------------- | --------------------------- | --------------------------------------- |
| `position`    | [`Point3`](#point3)         | The position component.                 |
| `orientation` | [`Quaternion`](#quaternion) | The orientation component (quaternion). |

###### Returns

[`Pose`](#pose)

#### Properties

| Property      | Modifier | Type                        | Description                                         |
| ------------- | -------- | --------------------------- | --------------------------------------------------- |
| `orientation` | `public` | [`Quaternion`](#quaternion) | The orientation component of the pose (quaternion). |
| `position`    | `public` | [`Point3`](#point3)         | The position component of the pose.                 |

______________________________________________________________________

### Quaternion

Represents a quaternion with w, x, y, z components, often used for 3D rotation.

This class is JSON-compatible: instances can be created directly from JSON.parse and serialized with JSON.stringify.

#### Example

```typescript
const q = new Quaternion(1.0, 0.0, 0.0, 0.0);
const json = JSON.stringify(q);               // '{"w":1,"x":0,"y":0,"z":0}'
const restored: Quaternion = JSON.parse(json); // { w: 1, x: 0, y: 0, z: 0 }

```

#### Constructors

##### Constructor

> **new Quaternion**(`w`, `x`, `y`, `z`): [`Quaternion`](#quaternion)

Creates a new [Quaternion](#quaternion) instance.

###### Parameters

| Parameter | Type     | Description                         |
| --------- | -------- | ----------------------------------- |
| `w`       | `number` | The scalar (real) component.        |
| `x`       | `number` | The X component of the vector part. |
| `y`       | `number` | The Y component of the vector part. |
| `z`       | `number` | The Z component of the vector part. |

###### Returns

[`Quaternion`](#quaternion)

#### Properties

| Property | Modifier | Type     | Description                         |
| -------- | -------- | -------- | ----------------------------------- |
| `w`      | `public` | `number` | The scalar (real) component.        |
| `x`      | `public` | `number` | The X component of the vector part. |
| `y`      | `public` | `number` | The Y component of the vector part. |
| `z`      | `public` | `number` | The Z component of the vector part. |

## Interfaces

### Actor

Handles actor related API calls. Provides methods to get and set the actor's pose within the virtual environment.

#### Methods

##### getCurrentPoseAsync()

> **getCurrentPoseAsync**(): `Promise`\<[`Pose`](#pose)>

Retrieves the current actor pose including position and orientation.

###### Returns

`Promise`\<[`Pose`](#pose)>

Resolves with the actor's pose.

##### setCurrentPoseAsync()

> **setCurrentPoseAsync**(`pose`): `Promise`\<`void`>

Sets the current actor pose including position and orientation.

###### Parameters

| Parameter | Type            | Description                    |
| --------- | --------------- | ------------------------------ |
| `pose`    | [`Pose`](#pose) | The pose to set for the actor. |

###### Returns

`Promise`\<`void`>

Resolves when the pose has been set successfully.

______________________________________________________________________

### BoundingBox

Handles bounding box related API calls.

#### Methods

##### getBoundingBoxAsync()

> **getBoundingBoxAsync**(`projectId`, `branchIds`): `Promise`\<{ `max`: [`Coordinate`](#coordinate); `min`: [`Coordinate`](#coordinate); }>

Retrieves the bounding box for specified branches within a project.

###### Parameters

| Parameter   | Type       | Description                                                     |
| ----------- | ---------- | --------------------------------------------------------------- |
| `projectId` | `string`   | The ID of the project containing the branches.                  |
| `branchIds` | `number`[] | An array of branch IDs for which to calculate the bounding box. |

###### Returns

`Promise`\<{ `max`: [`Coordinate`](#coordinate); `min`: [`Coordinate`](#coordinate); }>

Resolves with coordinates for the bounding box.

###### Example

```typescript
project.boundingBox.getBoundingBoxAsync('project123', [1, 2, 3]).then(function(bbox) {
    console.log('Bounding box coordinates:', bbox);
}).catch(function(error) {
    console.error('Error retrieving bounding box:', error);
});

```

______________________________________________________________________

### BranchOperations

Handles branch operations related API calls.

#### Methods

##### colorizeAsync()

> **colorizeAsync**(`projectId`, `branchIds`, `color`): `Promise`\<`string`>

Applies a colorize operation to the branches with the specified color.

###### Parameters

| Parameter   | Type       | Description                                        |
| ----------- | ---------- | -------------------------------------------------- |
| `projectId` | `string`   | The ID of the project containing the branches.     |
| `branchIds` | `number`[] | An array of branch IDs where the color is applied. |
| `color`     | `string`   | The color that will be applied to the branches.    |

###### Returns

`Promise`\<`string`>

Resolves when the colorizing is applied.

###### Example

```typescript
view.branchOperations.colorizeAsync('project123', [1, 2, 3], 'red').then(function(response) {
    console.log('Colorize status:', response);
}).catch(function(error) {
    console.error('Error colorizing:', error);
});

```

##### hideAsync()

> **hideAsync**(`projectId`, `branchIds`): `Promise`\<`string`>

Hides the specified branches in the project.

###### Parameters

| Parameter   | Type       | Description                                    |
| ----------- | ---------- | ---------------------------------------------- |
| `projectId` | `string`   | The ID of the project containing the branches. |
| `branchIds` | `number`[] | An array of branch IDs to hide.                |

###### Returns

`Promise`\<`string`>

A Promise that resolves when the branches have been hidden.

###### Example

```typescript
view.branchOperations.hideAsync('project123', [1, 2, 3]).then(function(response) {
    console.log('Hide status:', response);
}).catch(function(error) {
    console.error('Error hiding branches:', error);
});

```

##### hideEverythingAsync()

> **hideEverythingAsync**(`projectId`): `Promise`\<`string`>

Hides all branches in the project.

###### Parameters

| Parameter   | Type     | Description            |
| ----------- | -------- | ---------------------- |
| `projectId` | `string` | The ID of the project. |

###### Returns

`Promise`\<`string`>

A Promise that resolves when all branches have been hidden.

###### Example

```typescript
view.branchOperations.hideEverythingAsync('project123').then(function(response) {
    console.log('Hide everything status:', response);
}).catch(function(error) {
    console.error('Error hiding everything:', error);
});

```

##### resetAsync()

> **resetAsync**(`projectId`): `Promise`\<`string`>

Resets all branch operations in the project to their default state.

###### Parameters

| Parameter   | Type     | Description            |
| ----------- | -------- | ---------------------- |
| `projectId` | `string` | The ID of the project. |

###### Returns

`Promise`\<`string`>

A Promise that resolves when all operations have been reset.

###### Example

```typescript
view.branchOperations.resetAsync('project123').then(function(response) {
    console.log('Reset status:', response);
}).catch(function(error) {
    console.error('Error resetting operations:', error);
});

```

##### showAsync()

> **showAsync**(`projectId`, `branchIds`): `Promise`\<`string`>

Shows the specified branches in the project.

###### Parameters

| Parameter   | Type       | Description                                    |
| ----------- | ---------- | ---------------------------------------------- |
| `projectId` | `string`   | The ID of the project containing the branches. |
| `branchIds` | `number`[] | An array of branch IDs to show.                |

###### Returns

`Promise`\<`string`>

A Promise that resolves when the branches have been shown.

###### Example

```typescript
view.branchOperations.showAsync('project123', [1, 2, 3]).then(function(response) {
    console.log('Show status:', response);
}).catch(function(error) {
    console.error('Error showing branches:', error);
});

```

##### showEverythingAsync()

> **showEverythingAsync**(`projectId`): `Promise`\<`string`>

Shows all branches in the project.

###### Parameters

| Parameter   | Type     | Description            |
| ----------- | -------- | ---------------------- |
| `projectId` | `string` | The ID of the project. |

###### Returns

`Promise`\<`string`>

A Promise that resolves when all branches have been shown.

###### Example

```typescript
view.branchOperations.showEverythingAsync('project123').then(function(response) {
    console.log('Show everything status:', response);
}).catch(function(error) {
    console.error('Error showing everything:', error);
});

```

##### showOnlyAsync()

> **showOnlyAsync**(`projectId`, `branchIds`): `Promise`\<`string`>

Shows only the specified branches, hiding all others in the project.

###### Parameters

| Parameter   | Type       | Description                                    |
| ----------- | ---------- | ---------------------------------------------- |
| `projectId` | `string`   | The ID of the project containing the branches. |
| `branchIds` | `number`[] | An array of branch IDs to show exclusively.    |

###### Returns

`Promise`\<`string`>

A Promise that resolves when the operation is complete.

###### Example

```typescript
view.branchOperations.showOnlyAsync('project123', [1, 2, 3]).then(function(response) {
    console.log('Show only status:', response);
}).catch(function(error) {
    console.error('Error showing only branches:', error);
});

```

______________________________________________________________________

### Clipping

Handles clipping related API calls.

#### Methods

##### clipAsync()

> **clipAsync**(`min`, `max`): `Promise`\<{ `max`: [`Coordinate`](#coordinate); `min`: [`Coordinate`](#coordinate); }>

Applies a clipping box to the project to only display objects within specified coordinates.

###### Parameters

| Parameter | Type                        | Description                                  |
| --------- | --------------------------- | -------------------------------------------- |
| `min`     | [`Coordinate`](#coordinate) | The minimum coordinates of the clipping box. |
| `max`     | [`Coordinate`](#coordinate) | The maximum coordinates of the clipping box. |

###### Returns

`Promise`\<{ `max`: [`Coordinate`](#coordinate); `min`: [`Coordinate`](#coordinate); }>

Resolves when the clipping is applied.

###### Example

```typescript
project.clipping.clipAsync(
    { x: 0, y: 0, z: 0 },
    { x: 100, y: 100, z: 100 }
).then(function() {
    console.log('Clipping applied.');
}).catch(function(error) {
    console.error('Error applying clipping:', error);
});

```

##### resetClipAsync()

> **resetClipAsync**(): `Promise`\<`string`>

Removes any applied clipping boxes from the project.

###### Returns

`Promise`\<`string`>

Resolves when the clipping has been reset.

###### Example

```typescript
project.clipping.resetClipAsync().then(function() {
    console.log('Clipping reset.');
}).catch(function(error) {
    console.error('Error resetting clipping:', error);
});

```

______________________________________________________________________

### IApplyColorsAsyncArguments

#### Properties

| Property      | Type                            | Description                                  |
| ------------- | ------------------------------- | -------------------------------------------- |
| `colorGroups` | [`IColorGroup`](#icolorgroup)[] | Groups of colors and the tags they apply to. |

______________________________________________________________________

### IColorGroup

#### Properties

| Property    | Type       |
| ----------- | ---------- |
| `color`     | `string`   |
| `opacity`   | `number`   |
| `tagValues` | `string`[] |

______________________________________________________________________

### IExecuteCommandArguments

#### Properties

| Property       | Type     | Description                                  |
| -------------- | -------- | -------------------------------------------- |
| `commandInput` | `string` | Additional input for the command (optional). |
| `commandName`  | `string` | The name of the command to execute.          |

______________________________________________________________________

### IExecuteRequestArguments

#### Properties

| Property       | Type     |
| -------------- | -------- |
| `requestInput` | `string` |
| `requestName`  | `string` |

______________________________________________________________________

### ILabelPosition

Represents a 3D position for label placement.

#### Properties

| Property | Type     |
| -------- | -------- |
| `x`      | `string` |
| `y`      | `string` |
| `z`      | `string` |

______________________________________________________________________

### INotificationArguments

#### Properties

| Property            | Type     | Description                                       |
| ------------------- | -------- | ------------------------------------------------- |
| `notificationInput` | `string` | Additional input for the notification (optional). |
| `notificationName`  | `string` | The name of the notification to send.             |

______________________________________________________________________

### ISelectAndFlyToArguments

#### Properties

| Property   | Type     | Description                      |
| ---------- | -------- | -------------------------------- |
| `tagValue` | `string` | The tag of the object to select. |

______________________________________________________________________

### ISettingArguments

Copyright © Siemens 2026 ALL RIGHTS RESERVED.

#### Properties

| Property       | Type     |
| -------------- | -------- |
| `settingValue` | `string` |

______________________________________________________________________

### Label

Handles label related API calls.

#### Methods

##### addLabelAsync()

> **addLabelAsync**(`labelText`, `labelPosition`): `Promise`\<`string`>

Adds a label to the project at a specific position.

###### Parameters

| Parameter       | Type                                | Description                                          |
| --------------- | ----------------------------------- | ---------------------------------------------------- |
| `labelText`     | `string`                            | The text content of the label.                       |
| `labelPosition` | [`ILabelPosition`](#ilabelposition) | The 3D coordinates where the label should be placed. |

###### Returns

`Promise`\<`string`>

A Promise that resolves with the ID of the created label.

###### Example

```typescript
project.label.addLabelAsync("Important Component", { x: 10, y: 20, z: 30 }).then(function(labelId) {
    console.log('Label added with ID:', labelId);
}).catch(function(error) {
    console.error('Error adding label:', error);
});

```

##### removeLabelAsync()

> **removeLabelAsync**(`labelId`): `Promise`\<`string`>

Removes a label from the project.

###### Parameters

| Parameter | Type     | Description                    |
| --------- | -------- | ------------------------------ |
| `labelId` | `string` | The ID of the label to remove. |

###### Returns

`Promise`\<`string`>

A Promise that resolves when the label has been removed.

###### Example

```typescript
project.label.removeLabelAsync('1234').then(function() {
    console.log('Label removed.');
}).catch(function(error) {
    console.error('Error removing label:', error);
});

```

##### updateLabelPositionAsync()

> **updateLabelPositionAsync**(`labelId`, `labelPosition`): `Promise`\<`string`>

Updates the position of an existing label.

###### Parameters

| Parameter       | Type                                | Description                           |
| --------------- | ----------------------------------- | ------------------------------------- |
| `labelId`       | `string`                            | The ID of the label to update.        |
| `labelPosition` | [`ILabelPosition`](#ilabelposition) | The new 3D coordinates for the label. |

###### Returns

`Promise`\<`string`>

A Promise that resolves when the label position has been updated.

###### Example

```typescript
project.label.updateLabelPositionAsync('1234', { x: 15, y: 25, z: 35 }).then(function() {
    console.log('Label position updated.');
}).catch(function(error) {
    console.error('Error updating label position:', error);
});

```

##### updateLabelTextAsync()

> **updateLabelTextAsync**(`labelId`, `labelText`): `Promise`\<`string`>

Updates the text of an existing label.

###### Parameters

| Parameter   | Type     | Description                    |
| ----------- | -------- | ------------------------------ |
| `labelId`   | `string` | The ID of the label to update. |
| `labelText` | `string` | The new text for the label.    |

###### Returns

`Promise`\<`string`>

A Promise that resolves when the label text has been updated.

###### Example

```typescript
project.label.updateLabelTextAsync('1234', 'Updated Text').then(function() {
    console.log('Label text updated.');
}).catch(function(error) {
    console.error('Error updating label text:', error);
});

```

______________________________________________________________________

### Measurement

Handles measurement related API calls.

#### Methods

##### onRouteMeasurementCompleted()

> **onRouteMeasurementCompleted**(`handler`): `void`

Registers a handler for route measurement completed events.

###### Parameters

| Parameter | Type            | Description                                               |
| --------- | --------------- | --------------------------------------------------------- |
| `handler` | (`e`) => `void` | The function to call when route measurement is completed. |

###### Returns

`void`

##### startRouteMeasurementAsync()

> **startRouteMeasurementAsync**(): `Promise`\<`void`>

Begins a new route measurement within the project.

###### Returns

`Promise`\<`void`>

Resolves when the measurement has started.

###### Example

```typescript
project.measurement.startRouteMeasurementAsync().then(function() {
    console.log('Route measurement started.');
}).catch(function(error) {
    console.error('Error starting route measurement:', error);
});

```

##### stopRouteMeasurementAsync()

> **stopRouteMeasurementAsync**(): `Promise`\<`void`>

Stops the current route measurement and returns the results.

###### Returns

`Promise`\<`void`>

Resolves when the measurement has stopped.

###### Example

```typescript
project.measurement.stopRouteMeasurementAsync().then(function(result) {
    console.log('Route measurement stopped:', result);
}).catch(function(error) {
    console.error('Error stopping route measurement:', error);
});

```

______________________________________________________________________

### Project

Handles project related API calls.

#### Properties

| Property      | Type                          | Description       |
| ------------- | ----------------------------- | ----------------- |
| `boundingBox` | [`BoundingBox`](#boundingbox) | Bounding Box API. |
| `clipping`    | [`Clipping`](#clipping)       | Clipping API.     |
| `label`       | [`Label`](#label)             | Label API.        |
| `measurement` | [`Measurement`](#measurement) | Measurement API.  |
| `viewpoint`   | [`Viewpoint`](#viewpoint-1)   | Viewpoint API.    |

#### Methods

##### applyColorsAsync()

> **applyColorsAsync**(`args`): `Promise`\<`void`>

Applies coloring to objects based on their tags.

###### Parameters

| Parameter | Type                                                        | Description          |
| --------- | ----------------------------------------------------------- | -------------------- |
| `args`    | [`IApplyColorsAsyncArguments`](#iapplycolorsasyncarguments) | The color arguments. |

###### Returns

`Promise`\<`void`>

Resolves when colors have been applied successfully.

###### Example

```typescript
project.applyColorsAsync({
    colorGroups: [
        {
            color: 'red',
            opacity: 1.0,
            tagValues: ['pump1', 'pump2']
         }
    ]
}).then(function() {
    console.log('Colors applied to specified tags.');
}).catch(function(error) {
    console.error('Error applying colors:', error);
});

```

##### flyToSelectionAsync()

> **flyToSelectionAsync**(): `Promise`\<`void`>

Flies the camera to currently selected objects.

###### Returns

`Promise`\<`void`>

Resolves when the camera has moved to the selection.

###### Example

```typescript
project.flyToSelectionAsync().then(function() {
    console.log('Camera flown to selected objects.');
}).catch(function(error) {
    console.error('Error flying to selection:', error);
});

```

##### hideSelectionAsync()

> **hideSelectionAsync**(): `Promise`\<`void`>

Hides the selected objects.

###### Returns

`Promise`\<`void`>

Resolves when selected objects have been hidden.

###### Example

```typescript
project.hideSelectionAsync().then(function() {
    console.log('Selected objects have been hidden.');
}).catch(function(error) {
    console.error('Error hiding selected objects:', error);
});

```

##### jumpToHome()

> **jumpToHome**(): `void`

Resets the camera to its default home position.

###### Returns

`void`

###### Example

```typescript
project.jumpToHome();
console.log('Camera returned to home position.');

```

##### on()

> **on**(`eventName`, `handler`): `void`

###### Parameters

| Parameter   | Type                  |
| ----------- | --------------------- |
| `eventName` | `string`              |
| `handler`   | `MessageEventHandler` |

###### Returns

`void`

##### selectAndFlyToAsync()

> **selectAndFlyToAsync**(`args`): `Promise`\<`string`>

Selects an object by its tag and flies the camera to it.

###### Parameters

| Parameter | Type                                                    | Description              |
| --------- | ------------------------------------------------------- | ------------------------ |
| `args`    | [`ISelectAndFlyToArguments`](#iselectandflytoarguments) | The selection arguments. |

###### Returns

`Promise`\<`string`>

Resolves with the result of the selection operation.

###### Example

```typescript
project.selectAndFlyToAsync('pump1').then(function() {
    console.log('Camera flown to selected object.');
}).catch(function(error) {
    console.error('Error selecting or flying:', error);
});

```

##### showAllAsync()

> **showAllAsync**(): `Promise`\<`void`>

Makes all objects in the project visible.

###### Returns

`Promise`\<`void`>

Resolves when all objects are visible.

###### Example

```typescript
project.showAllAsync().then(function() {
    console.log('All objects are now visible.');
}).catch(function(error) {
    console.error('Error showing all objects:', error);
});

```

##### showOnlySelectionAsync()

> **showOnlySelectionAsync**(): `Promise`\<`void`>

Hides all unselected objects, showing only the selected ones.

###### Returns

`Promise`\<`void`>

Resolves when only selected objects are visible.

###### Example

```typescript
project.showOnlySelectionAsync().then(function() {
    console.log('Only selected objects are now visible.');
}).catch(function(error) {
    console.error('Error showing only selection:', error);
});

```

##### unselectAllAsync()

> **unselectAllAsync**(): `Promise`\<`void`>

Unselects all currently selected objects.

###### Returns

`Promise`\<`void`>

Resolves when all selections have been cleared.

###### Example

```typescript
project.unselectAllAsync().then(function() {
    console.log('All selections cleared.');
}).catch(function(error) {
    console.error('Error unselecting:', error);
});

```

______________________________________________________________________

### ProjectArguments

Defines the arguments required to load a project into the view.

#### Properties

| Property         | Type     | Description                                                                           |
| ---------------- | -------- | ------------------------------------------------------------------------------------- |
| `projectAddress` | `string` | The address of the project to load, typically a URL pointing to the project resource. |
| `token`          | `string` | The authorization token required to access the project.                               |

______________________________________________________________________

### Settings

Handles settings related API calls.

#### Methods

##### getCollisionAsync()

> **getCollisionAsync**(): `Promise`\<`string`>

Checks if collision detection is enabled.

###### Returns

`Promise`\<`string`>

Resolves with the current collision setting.

###### Example

```typescript
settings.getCollisionAsync().then(function(collision) {
    console.log('Current collision setting:', collision);
}).catch(function(error) {
    console.error('Error getting collision:', error);
});

```

##### getGravityAsync()

> **getGravityAsync**(): `Promise`\<`string`>

Retrieves the current gravity setting.

###### Returns

`Promise`\<`string`>

Resolves with the current gravity setting.

###### Example

```typescript
settings.getGravityAsync().then(function(gravity) {
    console.log('Current gravity setting:', gravity);
}).catch(function(error) {
    console.error('Error getting gravity:', error);
});

```

##### getViewTypeAsync()

> **getViewTypeAsync**(): `Promise`\<`string`>

Retrieves the current view type (e.g., first person or third person).

###### Returns

`Promise`\<`string`>

Resolves with the current view type.

###### Example

```typescript
settings.getViewTypeAsync().then(function(viewType) {
    console.log('Current view type:', viewType);
}).catch(function(error) {
    console.error('Error getting view type:', error);
});

```

##### onCollisionValueChanged()

> **onCollisionValueChanged**(`handler`): `void`

Registers a handler for collision value change events.

###### Parameters

| Parameter | Type                                                                | Description                                        |
| --------- | ------------------------------------------------------------------- | -------------------------------------------------- |
| `handler` | [`SettingValueChangeEventHandler`](#settingvaluechangeeventhandler) | The function to call when collision value changes. |

###### Returns

`void`

##### onGravityValueChanged()

> **onGravityValueChanged**(`handler`): `void`

Registers a handler for gravity value change events.

###### Parameters

| Parameter | Type                                                                | Description                                      |
| --------- | ------------------------------------------------------------------- | ------------------------------------------------ |
| `handler` | [`SettingValueChangeEventHandler`](#settingvaluechangeeventhandler) | The function to call when gravity value changes. |

###### Returns

`void`

##### onViewTypeValueChanged()

> **onViewTypeValueChanged**(`handler`): `void`

Registers a handler for view type value change events.

###### Parameters

| Parameter | Type                                                                | Description                                        |
| --------- | ------------------------------------------------------------------- | -------------------------------------------------- |
| `handler` | [`SettingValueChangeEventHandler`](#settingvaluechangeeventhandler) | The function to call when view type value changes. |

###### Returns

`void`

##### setCollisionAsync()

> **setCollisionAsync**(`isCollision`): `Promise`\<`string`>

Enables or disables collision detection in the project.

###### Parameters

| Parameter     | Type                                      | Description                  |
| ------------- | ----------------------------------------- | ---------------------------- |
| `isCollision` | [`ISettingArguments`](#isettingarguments) | The collision setting value. |

###### Returns

`Promise`\<`string`>

Resolves when the collision setting has been set.

###### Example

```typescript
settings.setCollisionAsync(true).then(function() {
    console.log('Collision detection enabled.');
}).catch(function(error) {
    console.error('Error setting collision:', error);
});

```

##### setGravityAsync()

> **setGravityAsync**(`isGravity`): `Promise`\<`string`>

Enables or disables gravity in the project environment.

###### Parameters

| Parameter   | Type                                      | Description                |
| ----------- | ----------------------------------------- | -------------------------- |
| `isGravity` | [`ISettingArguments`](#isettingarguments) | The gravity setting value. |

###### Returns

`Promise`\<`string`>

Resolves when the gravity setting has been set.

###### Example

```typescript
settings.setGravityAsync(true).then(function() {
    console.log('Gravity enabled.');
}).catch(function(error) {
    console.error('Error setting gravity:', error);
});

```

##### setViewTypeAsync()

> **setViewTypeAsync**(`viewTypeString`): `Promise`\<`string`>

Sets the view type for the project.

###### Parameters

| Parameter        | Type     | Description                                                    |
| ---------------- | -------- | -------------------------------------------------------------- |
| `viewTypeString` | `string` | The view type to set ('thirdpersonmode' or 'firstpersonmode'). |

###### Returns

`Promise`\<`string`>

Resolves when the view type has been set.

###### Example

```typescript
settings.setViewTypeAsync('thirdpersonmode').then(function() {
    console.log('View type set to third person.');
}).catch(function(error) {
    console.error('Error setting view type:', error);
});

```

______________________________________________________________________

### View

Handles view related API calls.

#### Properties

| Property           | Modifier   | Type                                    | Description                                                  |
| ------------------ | ---------- | --------------------------------------- | ------------------------------------------------------------ |
| `branchOperations` | `readonly` | [`BranchOperations`](#branchoperations) | Provides access to branch operations related to the view.    |
| `currentActor`     | `readonly` | [`Actor`](#actor)                       | Provides access to actor related operations within the view. |
| `settings`         | `readonly` | [`Settings`](#settings)                 | Provides access to the view's settings.                      |

#### Methods

##### executeCommandAsync()

> **executeCommandAsync**(`args`): `Promise`\<`void`>

Executes a predefined command within the view.

###### Parameters

| Parameter | Type                                                    | Description            |
| --------- | ------------------------------------------------------- | ---------------------- |
| `args`    | [`IExecuteCommandArguments`](#iexecutecommandarguments) | The command arguments. |

###### Returns

`Promise`\<`void`>

Resolves when the command has been executed successfully.

###### Example

```typescript
view.executeCommandAsync({
    commandName: 'ResetView',
    commandInput: 'optional_input'
}).then(function() {
    console.log('Command executed successfully.');
}).catch(function(error) {
    console.error('Error executing command:', error);
});

```

##### executeRequestAsync()

> **executeRequestAsync**(`args`): `Promise`\<`any`>

###### Parameters

| Parameter | Type                                                    |
| --------- | ------------------------------------------------------- |
| `args`    | [`IExecuteRequestArguments`](#iexecuterequestarguments) |

###### Returns

`Promise`\<`any`>

##### loadProjectAsync()

> **loadProjectAsync**(`args`): `Promise`\<[`Project`](#project)>

Loads a specified project into the initialized view.

###### Parameters

| Parameter | Type                                    | Description            |
| --------- | --------------------------------------- | ---------------------- |
| `args`    | [`ProjectArguments`](#projectarguments) | The project arguments. |

###### Returns

`Promise`\<[`Project`](#project)>

Resolves with the loaded instance.

###### Example

```typescript
view.loadProjectAsync({
    projectAddress: 'https://example.com/projects/123',
    token: 'your_access_token'
}).then(function(project) {
    console.info('Project loaded successfully:', project);
}).catch(function(error) {
    console.error('Error loading project:', error);
});

```

##### notifyAsync()

> **notifyAsync**(`args`): `Promise`\<`void`>

Sends a notification to all listeners within the view.

###### Parameters

| Parameter | Type                                                | Description                 |
| --------- | --------------------------------------------------- | --------------------------- |
| `args`    | [`INotificationArguments`](#inotificationarguments) | The notification arguments. |

###### Returns

`Promise`\<`void`>

Resolves when the notification has been sent successfully.

###### Example

```typescript
view.notifyAsync({
    notificationName: 'UpdateStatus',
    notificationInput: 'Updated to new status.'
}).then(function() {
    console.log('Notification sent successfully.');
}).catch(function(error) {
    console.error('Error sending notification:', error);
});

```

##### onBranchSelectionChanged()

> **onBranchSelectionChanged**(`handler`): `void`

Subscribes to branch selection change events in the viewer. When branches are selected or deselected in the 3D view, the provided handler will be invoked with details about the selection change action and the affected branch handles.

###### Parameters

| Parameter | Type                                  | Description                                                                                                                                                                                                                                                 |
| --------- | ------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `handler` | (`action`, `branchHandles`) => `void` | Callback function that receives selection change notifications. - `action`: The type of selection action that occurred (e.g., 'add', 'remove', 'clear') - `branchHandles`: Array of serialized branch IDs that were affected by the branch selection change |

###### Returns

`void`

###### Remarks

The format of each branch ID in the `branchHandles` array is a string "{projectId}/{branchId}"

##### onNotify()

> **onNotify**(`notificationName`, `notificationHandler`): `void`

###### Parameters

| Parameter             | Type                                          |
| --------------------- | --------------------------------------------- |
| `notificationName`    | `string`                                      |
| `notificationHandler` | [`NotificationHandler`](#notificationhandler) |

###### Returns

`void`

______________________________________________________________________

### Viewpoint

Handles viewpoint related API calls.

#### Methods

##### jumpToViewpointAsync()

> **jumpToViewpointAsync**(`projectId`, `viewpointId`): `Promise`\<`string`>

Moves the camera to a specific viewpoint identified by viewpointId.

###### Parameters

| Parameter     | Type     | Description                                     |
| ------------- | -------- | ----------------------------------------------- |
| `projectId`   | `string` | The ID of the project containing the viewpoint. |
| `viewpointId` | `string` | The ID of the viewpoint to jump to.             |

###### Returns

`Promise`\<`string`>

Resolves when the camera has moved to the viewpoint.

###### Example

```typescript
project.viewpoint.jumpToViewpointAsync('project123', 'viewpoint456').then(function() {
    console.log('Jumped to viewpoint.');
}).catch(function(error) {
    console.error('Error jumping to viewpoint:', error);
});

```

## Type Aliases

### NotificationHandler()

> **NotificationHandler** = (`e`) => `void`

#### Parameters

| Parameter | Type     |
| --------- | -------- |
| `e`       | `string` |

#### Returns

`void`

______________________________________________________________________

### SettingValueChangeEventHandler()

> **SettingValueChangeEventHandler** = (`e`) => `void`

#### Parameters

| Parameter | Type                    |
| --------- | ----------------------- |
| `e`       | [`Settings`](#settings) |

#### Returns

`void`

## Functions

### createViewAsync()

> **createViewAsync**(`viewOptions`): `Promise`\<[`View`](#view)>

Initializes a new 3D viewer within a specified container element.

#### Parameters

| Parameter                      | Type                                                                                           | Description                                                                                                                                                                           |
| ------------------------------ | ---------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `viewOptions`                  | { `backendAddress`: `string`; `targetElement?`: `HTMLElement`; `targetElementId?`: `string`; } | Configuration options for the view. Should contain `backendAddress` (server URL), and either `targetElementId` (DOM element ID) or `targetElement` (HTMLElement) to host the 3D view. |
| `viewOptions.backendAddress`   | `string`                                                                                       | The address of the backend page.                                                                                                                                                      |
| `viewOptions.targetElement?`   | `HTMLElement`                                                                                  | The target HTML element to host the viewer. Optional if `targetElementId` is provided.                                                                                                |
| `viewOptions.targetElementId?` | `string`                                                                                       | The ID of the target HTML element to host the viewer. Optional if `targetElement` is provided.                                                                                        |

#### Returns

`Promise`\<[`View`](#view)>

Resolves with the initialized instance.

#### Example

```typescript
walkinside.createViewAsync({
    backendAddress: 'https://example.com/walkinside-backend',
    targetElement: document.getElementById('wi-viewer-container')
}).then(function(view) {
    console.info('View created successfully.');
}).catch(function(error) {
    console.error('Error creating view:', error);
});

```

# Project and User Management APIs

Manages Walkinside project data.

[Download OpenAPI Specification](walkinside.webapi.json)

# UI Web Components

## Table of Contents

1. [Introduction](#introduction)
1. [wi-joystick-navigation](#wi-joystick-navigation)

______________________________________________________________________

## Introduction

Web components in WalkinsideJS provide reusable and customizable UI elements that can be easily integrated into your application. These components are designed to enhance user interaction and simplify the development process by encapsulating functionality and styling.

To use web components, ensure the following file is included:

```html


```

This file provide the necessary backend support for the web components.

## wi-joystick-navigation

The `wi-joystick-navigation` web component is a customizable joystick control designed for navigation within the Walkinside 3D viewer. It allows users to interact with the 3D environment using a virtual joystick.

### Usage

To use the `wi-joystick-navigation` component, include it in your HTML file as follows:

```html


```

### Features

- Provides a virtual joystick for navigation.
- Supports both mouse and touch interactions.
- Emits keyboard-like events for seamless integration with the Walkinside 3D viewer.

### Example

```html



    Joystick Navigation Example
    


    
      
    



```

In this example, the `wi-joystick-navigation` component is added alongside the Walkinside viewer container. The joystick can be used to navigate the 3D environment interactively.
# Workflow Canvas documentation

# Contact

For any inquiries regarding to Workflow Canvas, please contact:

[workflowcanvas.siemens@outlook.com](mailto:workflowcanvas.siemens@outlook.com)

- [Overview](overview.html)
- [Contact](contact.html)

# Overview

Workflow Canvas is a cloud-native IT/OT integration toolkit that enables Siemens and its eco-partners to develop integrated digital solutions faster, with web-based engineering, built-in 3D visualization, and container-based runtime ready to be deployed either on-premise or in the cloud. Its drag-and-drop interface provides an intuitive developer experience, while both IT and OT engineers can collaborate on the engineering of a plethora of devices, equipment and information systems. The resulting runtime is a cross-platform orchestrator (SPIDR) that comes with data integration options such as IIH and the Semantic Data Layer, offering highly flexible and customizable digital integration solutions to meet the rigorous demands of various industries.

Ready for your Workflow Canvas Journey? Simply [click here](https://wfc.siemens.cloud) to start!