Bulk Batch File Upload¶

Environment Paths on Linux

$ export AWS_ACCESS_KEY_ID=XXXXXXXXXXXXXXXXXXXX $ export AWS_SECRET_ACCESS_KEY=xxxxxxxxxxxxx/xxxxxxx/xxxxxxxxxxxxxxxxxx $ export AWS_SESSION_TOKEN=<>

Provide write permission¶

The tenantAdmin can access this API and provide write permission on folder or path, on the subtenants folder and on the root folder, except for the Time Series Import (TSI) folder.

POST /accessTokenPermissions

Content-Type: Content-Type: application/json

Request example:

{
  "subtenantId": "204a896c-a23a-11e9-a2a3-2a2ae2dbcce4",
  "path": "/myfolder1/myfolder2",
  "permission": "WRITE"
}

Response example:

{
  "subtenantId": "204a896c-a23a-11e9-a2a3-2a2ae2dbcce4",
  "id": "string",
  "path": "/myfolder1/myfolder2",
  "permission": "WRITE",
  "created": "2019-11-12T13:36:00.000Z"
}

Get a list of write permission folders¶

The tenantAdmin can get a list of all the folders that have the write permission. It is also possible to get a list of write permissions of the subtenants folders.

GET /accessTokenPermissions

Content-Type: application/json

Response example:

{
   "accessTokenPermissions": [
  {
    "subtenantId": "204a896c-a23a-11e9-a2a3-2a2ae2dbcce4",
    "id": "string",
    "path": "/myfolder1/myfolder2",
    "permission": "WRITE",
    "created": "2019-11-12T13:36:00.000Z"
  }
  ]
}

Get the list for write permission folders for the given id¶

You can get the list of folders that have the write permissions, for the given id. The tenantAdmin can also get the details of the request for subtenants. The subtenants can access this API to get details of the request belongs to their write folder.

GET /accessTokenPermissions/{id}

Content-Type: application/json

Response example:

{
  "subtenantId": "204a896c-a23a-11e9-a2a3-2a2ae2dbcce4",
  "id": "string",
  "path": "/myfolder1/myfolder2",
  "permission": "WRITE",
  "created": "2019-11-12T13:36:00.000Z""id": "0234sd34a23a-11e9-a2a3-2a2sdfw34ce4",
}

Revoke write permission¶

The TenantAdmin can access this API and provide delete permission on folders for the given id.

DELETE /accessTokenPermissions/{id}

Content-Type: application/json

Response example:

Status: 204 - deleted

Get the STS token with Write access¶

Request URL:

https://gateway.eu1.mindsphere.io/api/datalake/v3/generateAccessToken

Request example:

{
  "durationSeconds": 900,
  "permission": "WRITE"
}

Response example:

{
  "storageAccount": "dlbucketname",
  "storagePath": "data/ten=tenantname OR data/sub=subtenantId",
  "credentials": {
    "secretAccessKey": "AbCdEfGhIjKlMnOpQrStUvWxYz",
    "accessKeyId": "AbCdEfGhIjKlMnOpQrStUvWxYzAbCdEfGhIjKlMnOpQrStUvWxYz",
    "sessionToken": "AbCdEfGhIjKlMnOpQrStUvWxYzAbCdEfGhIjKlMnOpQrStUvWxYzAbCdEfGhIjKlMnOpQrStUvWxYzAbCdEfGhIjKlMnOpQrStUvWxYzAbCdEfGhIjKlMnOpQrStUvWxYzAbCdEfGhIjKlMnOpQrStUvWxYz"
  },
  "durationSeconds": 900,
  "permission": "WRITE"
}

Command and Output¶

The following command is used for Bulk upload:

aws s3 cp s3://ms-coldstorage-iot-integ-eu-central-1/000013bb5ed740f9950504854367e12a s3://estbuckettimeseries/ten=dide2/000013bb5ed740f9950504854367e12a --recursive

Completed 1.7 KiB/1.7 KiB (812 Bytes/s) with 1 file(s) remaining
copy: s3://ms-coldstorage-iot-integ-eu-central-1/000013bb5ed740f9950504854367e12a/propSetName=SPSNFRTestFields/date=2019-04-05/0900.parquet to s3://estbuckettimeseries/ten=dide2/000013bb5ed740f995050485436
7e12a/propSetName=SPSNFRTestFields/date=2019-04-05/0900.parquet

STS Token Additional API¶

Allows users to request temporary, limited-privilege AWS credentials to get read-only or write-only access on the URI returned in the response.

• Generated access credentials can be used to access all the paths provided in the request with their respective permissions.
• Path field is mandatory if any permission is provided in the request or vice-versa.
• LIST permission is independent of the path and permission fields in the request. By default, LIST objects permission will be provided on the root prefix ("/") of environment or subtenant.
• If path and permission, both fields are not provided in the request then “READ” permission will be given on the root prefix ("/") of environment or subtenant.
• An access credential requested for a given path also automatically gives access to all sub-paths of the requested path. For example, if an access token is requested for path “/a” and there are sub-paths “/a/b” and “/a/b/c”, the token allows to access those too.
• 1024 characters are supported as maximum length for path field. If the path length is shorter, then multiple accesses on different prefixes are supported in the same request. Otherwise, a single path prefix of max 1024 characters will only be supported. The number of prefixes supported will depend on the path length/permission specified in the request.

POST /generateAccessCredentials

Content-Type: application/json

Request example:

{
  "durationSeconds": 900,
  "accesses": [
    {
      "path": "/myfolder1/folder2",
      "subtenantId": "204a896c-a23a-11e9-a2a3-2a2ae2dbcce4",
      "permissions": [
        "READ",
        "WRITE",
        "DELETE"
      ]
    }
  ]
}

Response example:

{
  "credentials": {
    "secretAccessKey": "AbCdEfGhIjKlMnOpQrStUvWxYz",
    "accessKeyId": "AbCdEfGhIjKlMnOpQrStUvWxYzAbCdEfGhIjKlMnOpQrStUvWxYz",
    "sessionToken": "AbCdEfGhIjKlMnOpQrStUvWxYzAbCdEfGhIjKlMnOpQrStUvWxYzAbCdEfGhIjKlMnOpQrStUvWxYzAbCdEfGhIjKlMnOpQrStUvWxYzAbCdEfGhIjKlMnOpQrStUvWxYzAbCdEfGhIjKlMnOpQrStUvWxYz"
  },
  "storageAccount": "dlbucketname",
  "durationSeconds": 900,
  "accesses": [
    {
      "path": "/myfolder1/folder2",
      "subtenantId": "204a896c-a23a-11e9-a2a3-2a2ae2dbcce4",
      "permissions": [
        "READ",
        "WRITE",
        "DELETE"
      ]
    }
  ]
}