Bulk Operations¶
API MAY provide bulk operations [900]¶
Sometimes, you might want to give users of your API the option to execute an operation on multiple resources via a single call, e.g., due to atomicity or due to performance benefits. In such cases, you MAY offer URLs for bulk operations as part of the API.
Here we focus on bulk processing that targets the execution of a single type of operation per call—as opposed to mixed types of operations per call, sometimes referred to as batch processing. Furthermore, we focus on bulk processing of resources of the same type in a single call—i.e., on a collection resource—and on CRUD operations.
As bulk operations increase the complexity, only add them if required. You SHOULD NOT offer bulk operations as the only option to process a specific resource type. Instead, you SHOULD provide URLs for processing single resources before providing bulk operations. Furthermore, you SHOULD define limits (e.g. for maximum number of REST resources) for each bulk operation.
Bulk operations might be combined with long-running operations. Nonetheless, these concepts should be clearly separated, and the following sections focus on the concept of bulk operations.
Bulk endpoints SHOULD use the collection resource's path [900.1]¶
If you decide to offer bulk operations for a resource type, you SHOULD use the URL of the according collection resource, for example:
/devices
Bulk endpoints SHOULD use the media type application/vnd.siemens.bulk+json
[900.2]¶
A bulk operation (except GET
) SHOULD be indicated explicitly via the media type application/vnd.siemens.bulk+json
. Example:
POST /devices HTTP/1.1
Content-Type: application/vnd.siemens.bulk+json
Accept: application/json
The benefit of using a dedicated media type for bulk operations is that the operation can easily be distinguished from standard operations. In contrast, identifying a bulk operation via a different body would require parsing the body. Using a dedicated bulk URL including the verb would impact the maturity of the REST API.
Bulk endpoints SHOULD use the HTTP verbs GET
, POST
, PATCH
, PUT
, or DELETE
[900.3]¶
Bulk endpoints SHOULD use the same HTTP verbs for CRUD operations as endpoints for single resources.
Use GET
to fetch a resource collection (see 800.2).
Use POST
to add multiple new resources to a resource collection. Example:
POST /devices HTTP/1.1
Content-Type: application/vnd.siemens.bulk+json
Accept: application/json
{
"data": [
{
"name": "My Device",
"deviceType": {
"id": "hvac",
"name": "HVAC device"
},
"dimension": {
"width": 1.3,
"height": 2.52,
"depth": 0.9
},
"owner": "Werner Inc.",
"tags": [
"alarming",
"failsafe",
"redundant"
]
},
{
"name": "My Other Device",
"deviceType": {
"id": "hvac",
"name": "HVAC device"
},
"dimension": {
"width": 1.5,
"height": 2.3,
"depth": 0.9
}
}
]
}
Note that the data
property is an array.
Use PATCH
to update fields of multiple resources of a resource collection. Example:
PATCH /devices HTTP/1.1
Content-Type: application/vnd.siemens.bulk+json
Accept: application/json
{
"data": [
{
"id": "550e8400-e29b-41d4-a716-446655440000",
"owner": null
},
{
"id": "14d59c5f-1e97-4907-a1fd-50f370d31b15",
"dimension": {
"width": 1.35
},
"tags": [
"failsafe",
"redundant"
]
}
]
}
Note that the data
property is an array and that the id
property is used to identify the resources to be patched. The request uses JSON Merge Patch as described in 802.1.
Use PUT
to update multiple resources of a resource collection by replacing them. Example:
PUT /devices HTTP/1.1
Content-Type: application/vnd.siemens.bulk+json
Accept: application/json
{
"data": [
{
"id": "550e8400-e29b-41d4-a716-446655440000",
"name": "My Device",
"deviceType": {
"id": "hvac",
"name": "HVAC device"
},
"dimension": {
"width": 1.3,
"height": 2.52,
"depth": 0.9
},
"owner": "Werner Inc.",
"tags": [
"alarming",
"failsafe",
"redundant"
]
},
{
"id": "14d59c5f-1e97-4907-a1fd-50f370d31b15",
"name": "My Other Device",
"deviceType": {
"id": "hvac",
"name": "HVAC device"
},
"dimension": {
"width": 1.5,
"height": 2.3,
"depth": 0.9
}
}
]
}
Note that the data
property is an array and that the id
property is used to identify the resources to be replaced.
Use DELETE
to delete multiple resources of a resource collection. Example:
DELETE /devices HTTP/1.1
Content-Type: application/vnd.siemens.bulk+json
Accept: application/json
{
"data": [
{
"id": "550e8400-e29b-41d4-a716-446655440000"
},
{
"id": "14d59c5f-1e97-4907-a1fd-50f370d31b15"
}
]
}
Note that the data
property is an array and that the id
property is used to identify the resources to be deleted.
Trying to delete an already deleted object in a bulk operation SHOULD NOT be considered an error.
Bulk endpoints SHOULD be atomic [900.4]¶
For successful atomic bulk operations, you SHOULD return the same HTTP status codes as defined for the respective common operation on single resources.
Meaning of Atomic
Atomic just means that the whole bulk operation either succeeds or fails, there is no partial success in atomic operations.
Example: Response to a successful bulk PATCH
operation
HTTP/1.1 200 OK
Content-Type: application/json
{
"data": [
{
"id": "550e8400-e29b-41d4-a716-446655440000",
"name": "My Device",
"deviceType": {
"id": "hvac",
"name": "HVAC device"
},
"dimension": {
"width": 1.3,
"height": 2.52,
"depth": 0.9
},
"owner": "Werner Inc.",
"tags": [
"alarming",
"failsafe",
"redundant"
]
},
{
"id": "14d59c5f-1e97-4907-a1fd-50f370d31b15",
"name": "My Other Device",
"deviceType": {
"id": "hvac",
"name": "HVAC device"
},
"dimension": {
"width": 1.5,
"height": 2.3,
"depth": 0.9
},
"owner": null,
"tags": null
}
]
}
Example: Response to a successful bulk PATCH
operation without a response body
HTTP/1.1 204 No Content
In case of an error, apply the error reporting guidelines. Individual details about failed items MAY be added via entries in the errors
array. The overall HTTP status code SHOULD be aligned with the individual errors.
Example: Response to a bulk DELETE
operation with errors
HTTP/1.1 403 Forbidden
Content-Type: application/json
{
"errors": [
{
"status": 403,
"title": "Forbidden",
"source": {
"resourceId": "550e8400-e29b-41d4-a716-446655440000"
}
},
{
"status": 403,
"title": "Forbidden",
"source": {
"resourceId": "14d59c5f-1e97-4907-a1fd-50f370d31b15"
}
}
]
}
The example above demonstrates the response to a request for the deletion of multiple resources, where the deletion of two resources has failed and—due to the atomicity—no resource has been deleted at all.
Sometimes, the API client needs to map items of the response to items of the request, even if they do not have an ID at the time of the request, e.g., in order to map IDs for newly created resources or to map errors. In this case, you MUST provide a means to map items accordingly. If the success response contains an item for each item in the request, you SHOULD use the same order.
Example: Response to a successful bulk POST
operation
HTTP/1.1 200 OK
Content-Type: application/json
{
"data": [
{
"id": "550e8400-e29b-41d4-a716-446655440000",
"name": "My Device",
"deviceType": {
"id": "hvac",
"name": "HVAC device"
},
"dimension": {
"width": 1.3,
"height": 2.52,
"depth": 0.9
},
"owner": "Werner Inc.",
"tags": [
"alarming",
"failsafe",
"redundant"
]
},
{
"id": "14d59c5f-1e97-4907-a1fd-50f370d31b15",
"name": "My Other Device",
"deviceType": {
"id": "hvac",
"name": "HVAC device"
},
"dimension": {
"width": 1.5,
"height": 2.3,
"depth": 0.9
},
"owner": null,
"tags": null
}
]
}
Note that the response contains two items in the data
array. Their order is aligned with the order of the items to be created in the request, and hence it is not required to add any further mapping.
Example: Response to a bulk POST
operation with errors
HTTP/1.1 403 Forbidden
Content-Type: application/json
{
"errors": [
{
"status": 403,
"title": "Forbidden",
"source": {
"pointer": "data/0/dimension/width"
}
},
{
"status": 403,
"title": "Forbidden",
"source": {
"pointer": "data/15/dimension/width"
}
}
]
}
The example above demonstrates the response to a request for the creation of multiple resources, where the creation of two resources has failed and—due to the atomicity—no resource has been created at all. As the items in the request do not have an ID yet, they are referenced in the error response via the source.pointer
property. These two items have been the 1st and 16th item, respectively, in the request.
You MAY add additional mapping info to the response of a successful atomic bulk operation.
Example: Response to a successful bulk POST
operation
HTTP/1.1 200 OK
Content-Type: application/json
{
"data": [
{
"id": "550e8400-e29b-41d4-a716-446655440000",
"meta": {
"sourcePointer": "data/0"
},
"name": "My Device",
"deviceType": {
"id": "hvac",
"name": "HVAC device"
},
"dimension": {
"width": 1.3,
"height": 2.52,
"depth": 0.9
},
"owner": "Werner Inc.",
"tags": [
"alarming",
"failsafe",
"redundant"
]
},
{
"id": "14d59c5f-1e97-4907-a1fd-50f370d31b15",
"meta": {
"sourcePointer": "data/1"
},
"name": "My Other Device",
"deviceType": {
"id": "hvac",
"name": "HVAC device"
},
"dimension": {
"width": 1.5,
"height": 2.3,
"depth": 0.9
},
"owner": null,
"tags": null
}
]
}
Note that the response contains two items in the data
array. Their order is aligned with the order of the items to be created in the request, and hence it is not required to add any further mapping. Although not required, an additional mapping to the request is added explicitly via the meta.sourcePointer
property for convenience.
Bulk endpoints MAY be non-atomic [900.5]¶
Note that the introduction of non-atomic (bulk) endpoints, and hence of partial success, will increase the complexity of your API.
For successful non-atomic bulk operations, you SHOULD return the same HTTP status codes as defined for the respective common operation on single resources.
In case of a basic error that prevents processing any item, for example a malformed request, apply the error reporting guidelines.
In case of partial success, you SHOULD return the same HTTP status codes as defined for the respective common operation on single resources. In the response body, you MUST identify the items that could not be processed. You SHOULD consider the result of a bulk operation as a partial success if the basic request was understood, potential prerequisites have been checked successfully, and the service started processing the items of the request. This includes situations where the items collection of the request is actually empty or all items failed.
An API client MUST assume that items that are not reported as failed in the response body have been successfully processed. You MAY include information about the items that have been successfully processed, e.g., you might return the IDs or additional data of newly created items.
Example: Response to a non-atomic bulk DELETE
operation with partial success
HTTP/1.1 200 OK
Content-Type: application/json
{
"errors": [
{
"status": 403,
"title": "Forbidden",
"source": {
"resourceId": "550e8400-e29b-41d4-a716-446655440000"
}
},
{
"status": 403,
"title": "Forbidden",
"source": {
"resourceId": "14d59c5f-1e97-4907-a1fd-50f370d31b15"
}
}]
}
The example above demonstrates the response to a request for the deletion of multiple resources, where the deletion of two resources has failed. Although not listed explicitly, the deletion of the other resources has been successful due to the overall HTTP status code. Additionally, the successfully deleted resources could have been listed explicitly in a top-level data
property. Note that the items are referenced via the id
property.
Example: Response to a non-atomic bulk PATCH
operation with partial success
HTTP/1.1 200 OK
Content-Type: application/json
{
"data": [
{
"id": "550e8400-e29b-41d4-a716-446655440000",
"name": "My Device",
"deviceType": {
"id": "hvac",
"name": "HVAC device"
},
"dimension": {
"width": 1.3,
"height": 2.52,
"depth": 0.9
},
"owner": "Werner Inc.",
"tags": [
"alarming",
"failsafe",
"redundant"
]
},
{
"id": "14d59c5f-1e97-4907-a1fd-50f370d31b15",
"name": "My Other Device",
"deviceType": {
"id": "hvac",
"name": "HVAC device"
},
"dimension": {
"width": 1.5,
"height": 2.3,
"depth": 0.9
},
"owner": null,
"tags": null
}
],
"errors": [
{
"status": 404,
"title": "Not Found",
"source": {
"resourceId": "4e6d7Fcd-c44b-4c17-b44f-479686e4df62"
}
},
{
"status": 404,
"title": "Not Found",
"source": {
"resourceId": "de77fdb5-9240-4e23-9a21-d5a12284e0a6"
}
}]
}
The example above demonstrates the response to a request for updating multiple resources, where the update of two resources has failed. The successfully updated resources are listed explicitly. Note that the items are referenced via the resourceId
property.
Note
Responses with partial success are an exception to the rule that the top-level properties data
and errors
MUST NOT coexist in the same response document (see 101.2).
Sometimes, the API client needs to map items of the response to items of the request, even if they do not have an ID at the time of the request, e.g., in order to map IDs for newly created resources or to map errors. In this case, you MUST provide a means to map items accordingly.
Example: Response to a non-atomic bulk POST
operation with partial success
HTTP/1.1 200 OK
Content-Type: application/json
{
"data": [
{
"id": "550e8400-e29b-41d4-a716-446655440000",
"meta": {
"sourcePointer": "data/1"
},
"name": "My Device",
"deviceType": {
"id": "hvac",
"name": "HVAC device"
},
"dimension": {
"width": 1.3,
"height": 2.52,
"depth": 0.9
},
"owner": "Werner Inc.",
"tags": [
"alarming",
"failsafe",
"redundant"
]
},
{
"id": "14d59c5f-1e97-4907-a1fd-50f370d31b15",
"meta": {
"sourcePointer": "data/3"
},
"name": "My Other Device",
"deviceType": {
"id": "hvac",
"name": "HVAC device"
},
"dimension": {
"width": 1.5,
"height": 2.3,
"depth": 0.9
},
"owner": null,
"tags": null
}
],
"errors": [
{
"status": 400,
"title": "Bad Request",
"description": "Value must be greater than 0.",
"source": {
"pointer": "data/0/dimension/width"
}
},
{
"status": 400,
"title": "Bad Request",
"description": "Value must be greater than 0.",
"source": {
"pointer": "data/2/dimension/width"
}
},
{
"status": 400,
"title": "Bad Request",
"description", "Value must not be null.",
"source": {
"pointer": "data/2/name"
}
}]
}
The example above demonstrates the response to a request for the creation of multiple resources, where the creation of two resources has failed. These two items have been the 1st and 3rd item, respectively, in the request. As the items in the request do not have an ID yet, they are referenced in the errors
array via the source.pointer
property. Two resources, on the other hand, could be created successfully. These have been the 2nd and 4th item, respectively, in the request. Note that they are referenced via the meta.sourcePointer
property. Furthermore, each newly created resource has been assigned an ID, which is available via the id
property.