Object and Folder Operations¶

Object Operations via App Gateway¶

The user can upload the files and the associated metadata separately with a single API call. To upload the file, user needs to generate pre-signed URL i.e. SAS URL. It is possible to use multiple times during the validity period, minimum of 15 minutes. The requirement is to eliminate the use of pre-signed URL with prolonged validity periods. Uploading and downloading the files using APIs will provide additional flexibility in handling other scenarios like secure lock and managing object lifecycle.

This API endpoint allows the user to upload an object to a given path along with the metadata. The validations on the metadata is done as per the configurations done by the Integrated Data Lake Administrator. Validations on the metadata is done and the file upload is completed. Once the file upload is completed, the metadata is saved.If the file upload is complete and the metadata update fails, then file is only visible to Integrated Data Lake Administrator if the Attribute Based Access Control (ABAC) rules don't allow reading of the file without the object metadata.

Object operations via app gateway are described below:

• Upload an object via app gateway with metadata

  APIs can upload the files up to 100MB, multipart upload is provided later. In the API endpoint, if the path contains special characters, they are encoded. Encode only the words that contains special characters. Special characters supported in file and folder name are described below:

  • Alphanumeric characters (word characters: A-Z, a-z, 0-9 and underscore"_")
  • Space (" "), except at the start and end
  • Period (".")
  • Exclamation mark ("!")
  • Asterisk ("*")
  • Single quote ("'")
  • Ampersand ("&")
  • Dollar sign ("$")
  • At symbol ("@")
  • Colon (":")
  • Plus sign ("+")
  • Comma (",")
  • Square brackets ("[" and "]")
  • Less than ("<") and greater than (">") symbols
  • Tilde ("~")
  • Double quote (""")
  • Hash ("#")
  • Vertical bar ("|")
  • Question mark ("?")
  • Curly braces ("{" and "}")
  • Caret ("^")
  • Parentheses ("(" and ")")
  • Underscore ("_") but not at the beginning or at end, only for folder name
  • Hyphen ("-")
  • Equal sign ("=")

• Download Object via App Gateway

  Attribute Based Access Control (ABAC) are used to download an object in the path is permitted as per the policy created for the user by tenant admin. For more information on ABAC, refer to Attribute Based Access Control (ABAC) configuration.

Folder Operations via App Gateway¶

The user can create the folders and the associate metadata with a single API call. This API endpoint allows the user to create a folder in a given path along with the metadata. The validations on the metadata is done as per the configurations.

After the validation on the metadata is done, a folder is successfully created. Once the folder creation is completed, the metadata is saved. If a folder creation is completed and the metadata update fails, then a folder will only be visible to the Integrated Data Lake Administrator, if the Attribute Based Access Control (ABAC) rules don't allow access on a folder without the metadata.

• Create empty folder with Metadata

  User can create an empty folder for the specified path and optional metadata values as per the metadata keys defined by the Integrated Data Lake admin.

  Note

  If folder names contains special characters, no encoding is required as it is part of request body.

  Special characters supported in the folder name are described below:

  • Alphanumeric characters (word characters: A-Z, a-z, 0-9, and underscore "_")
  • Space (" "), except at the start and end
  • Period (".")
  • Exclamation mark ("!")
  • Asterisk ("*")
  • Single quote ("'")
  • Ampersand ("&")
  • Dollar sign ("$")
  • At symbol ("@")
  • Colon (":")
  • Plus sign ("+")
  • Comma (",")
  • Square brackets ("[" and "]")
  • Less than ("<") and greater than (">") symbols
  • Tilde ("~")
  • Double quote (""")
  • Hash ("#")
  • Vertical bar ("|")
  • Question mark ("?")
  • Curly braces ("{" and "}")
  • Caret ("^")
  • Parentheses ("(" and ")")
  • Hyphen ("-")
  • Equal sign ("=")

• Delete an empty Folder

  Delete an empty folder along with the metadata key values from the specified path.

  Note

  Operation is performed only on leaf folder. There is no impact on folder hierarchy.