CloudOp

Summary

Contains IO methods that are cloud specific.

Discussion

This class cannot be instantiated directly. An instance of this class is returned by the AIO object's cloud property.

Methods

`clearcache(path)`

Clears in-memory directory listings, file properties, and BLOB region cache. Call this method when an external process updates folders or files in the cloud store that were previously accessed.

This method is only applicable to cloud stores.

Name	Explanation	Data type
path	The relative path from the current working directory or an absolute path (absolute `vsi` path for cloud).	String

cloud_io = AIO(r"C:\data\datacloud.acs")
cloud_io.cloud.clearcache(r"aio")

`getbucketname()`

Gets the name of a connected bucket or container.

This method is only applicable to cloud stores.

Return value

Data type	Explanation
String	The name of the bucket or container.

cloud_io = AIO(r"C:\data\datacloud.acs")
cloud_io.cloud.getbucketname()

`getendpoint()`

Gets the cloud endpoint specified in the connection.

This method is only applicable to cloud stores.

Return value

Data type	Explanation
String	The cloud endpoint if available.

cloud_io = AIO(r"C:\data\datacloud.acs")
cloud_io.cloud.getendpoint()

`getfile(src, dst, {options})`

Downloads a file from the cloud.

This method is only applicable to cloud stores.

Name Explanation Data type

Name	Explanation	Data type
src	The relative path from the current working directory or an absolute `vsi` cloud path.	String
dst	The local, or target, path of the file.	String
options (Optional)	Specifies the options that will be used. `RECURSIVE`—Subdirectories and their contents will be recursively copied. This option is only applicable for folder operations. `SYNC_STRATEGY`—The criteria that will be used for overwriting if the target file exists. `NUM_THREADS`—The number of threads that will be used for parallel file copying. `CHUNK_SIZE`—The maximum chunk size (in bytes) that will be used to split large objects when uploading or downloading. Syntax—`RECURSIVE=[YES\|NO], SYNC_STRATEGY=[TIMESTAMP\|ETAG\|OVERWRITE], NUM_THREADS=(integer) CHUNK_SIZE=(integer) [x-amz-\|x-goog-\|x-ms-*=(value)]` Example—`{'RECURSIVE':'YES', 'SYNC_STRATEGY':'OVERWRITE', 'NUM_THREADS':'10', 'CHUNK_SIZE':'8'}` The default value is None.	Dictionary

src

The relative path from the current working directory or an absolute vsi cloud path.

String

dst

The local, or target, path of the file.

String

options

(Optional)

Specifies the options that will be used.

RECURSIVE—Subdirectories and their contents will be recursively copied. This option is only applicable for folder operations.
SYNC_STRATEGY—The criteria that will be used for overwriting if the target file exists.
NUM_THREADS—The number of threads that will be used for parallel file copying.
CHUNK_SIZE—The maximum chunk size (in bytes) that will be used to split large objects when uploading or downloading.

Example—{'RECURSIVE':'YES', 'SYNC_STRATEGY':'OVERWRITE', 'NUM_THREADS':'10', 'CHUNK_SIZE':'8'}

The default value is None.

Dictionary

Return value

Data type	Explanation
Boolean	True if the file download is successful; otherwise, False.

cloud_io = AIO(r"C:\data\datacloud.acs")
cloud_io.cloud.getfile(r'utf8_test.json', r"c:\data\datafile.json")

`getmetadata(path, {domain})`

Gets the metadata of a file or folder.

This method is only applicable to cloud stores.

Name Explanation Data type

path

The relative path from the current working directory or an absolute path (absolute vsi path for cloud).

String

domain

(Optional)

The metadata domain, for example, (https://gdal.org/api/cpl.html#_CPPv418VSIGetFileMetadataPKcPKc12CSLConstList).

The default value is None.

String

Return value

Data type	Explanation
Dictionary	A dictionary of the metadata information.

cloud_io = AIO(r"C:\data\datacloud.acs")
cloud_io.cloud.getmetadata(r'aio')

`getobjectstorepath()`

Gets the root path of the connection.

This method is only applicable to cloud stores.

Return value

Data type	Explanation
String	The `vsi` path to the object store.

cloud_io = AIO(r"C:\data\datacloud.acs")
cloud_io.cloud.getobjectstorepath()

`getproperties(path)`

Gets the header properties.

Name	Explanation	Data type
path	The relative path from the current working directory or an absolute path (absolute `vsi` path for cloud).	String

Return value

Data type	Explanation
Dictionary	A dictionary of header properties.

cloud_io = AIO(r"C:\data\datacloud.acs")
cloud_io.cloud.getproperties(r'aio')

`getprovidername()`

Gets the cloud provider name.

This method is only applicable to cloud stores.

Return value

Data type	Explanation
String	The cloud provider name.

cloud_io = AIO(r"C:\data\datacloud.acs")
cloud_io.cloud.getprovidername()

`getprovideroptions()`

Gets the provider options specified in the connection.

This method is only applicable for cloud stores.

Return value

Data type	Explanation
List	The provider options, if available.

cloud_io = AIO(r"C:\data\datacloud.acs")
cloud_io.cloud.getprovideroptions()

`getregion()`

Gets the cloud region specified in the connection.

This method is only applicable to cloud stores.

Return value

Data type	Explanation
String	The cloud region, if available.

cloud_io = AIO(r"C:\data\datacloud.acs")
cloud_io.cloud.getregion()

`getsignedurl(path, expiry, {upload})`

Generates a temporary HTTP presigned URL.

This method is only applicable to cloud stores.

Name Explanation Data type

path

The relative path from the current working directory or an absolute path (absolute vsi path for cloud).

String

expiry

The expiry time in seconds.

Integer

upload

(Optional)

Specifies whether the URL will be generated for upload. If True, the URL will be generated.

The default value is False.

Boolean

Return value

Data type	Explanation
String	The HTTP signed URL.

cloud_io = AIO(r"C:\data\datacloud.acs")
cloud_io.cloud.getsignedurl(r'data.json', 5)

`putfile(src, dst, {options})`

Uploads a file to cloud.

This method is only applicable for cloud.

Name Explanation Data type

Name	Explanation	Data type
src	The local (source) path of the file.	String
dst	The relative path from the current working directory or an absolute `vsi` cloud path.	String
options (Optional)	Specifies the options that will be used. `RECURSIVE`—Subdirectories and their contents will be recursively copied. This option is only applicable to folder operations. `SYNC_STRATEGY`—The criteria that will be used for overwriting if the target file exists. `NUM_THREADS`—The number of threads that will be used for parallel file copying. `CHUNK_SIZE`—The maximum chunk size (in bytes) that will be used to split large objects when uploading or downloading. Syntax—`RECURSIVE=[YES\|NO], SYNC_STRATEGY=[TIMESTAMP\|ETAG\|OVERWRITE], NUM_THREADS=(integer) CHUNK_SIZE=(integer) [x-amz-\|x-goog-\|x-ms-*=(value)]` Example—`{'RECURSIVE':'YES', 'SYNC_STRATEGY':'OVERWRITE', 'NUM_THREADS':'10', 'CHUNK_SIZE':'8'}` The default value is None.	Dictionary

src

The local (source) path of the file.

String

dst

The relative path from the current working directory or an absolute vsi cloud path.

String

options

(Optional)

Specifies the options that will be used.

RECURSIVE—Subdirectories and their contents will be recursively copied. This option is only applicable to folder operations.
SYNC_STRATEGY—The criteria that will be used for overwriting if the target file exists.
NUM_THREADS—The number of threads that will be used for parallel file copying.
CHUNK_SIZE—The maximum chunk size (in bytes) that will be used to split large objects when uploading or downloading.

Example—{'RECURSIVE':'YES', 'SYNC_STRATEGY':'OVERWRITE', 'NUM_THREADS':'10', 'CHUNK_SIZE':'8'}

The default value is None.

Dictionary

Return value

Data type	Explanation
Boolean	True if the file upload is successful; otherwise, False.

cloud_io = AIO(r"C:\data\datacloud.acs")
cloud_io.cloud.putfile(r"C:\data\data_file.json", r'data_folder\data_file_new.json')

`setmetadata(path, {domain}, {metadata}, {options})`

Sets the metadata of a file or folder.

This method is only applicable to cloud stores.

Name	Explanation	Data type
path	The relative path from the current working directory or an absolute path (absolute `vsi` path for cloud).	String
domain (Optional)	The metadata domain, for example, `(https://gdal.org/api/cpl.html#_CPPv418VSISetFileMetadataPKc12CSLConstListPKc12CSLConstList)`. The default value is None.	String
metadata (Optional)	The metadata that will be set. The default value is None.	Dictionary
options (Optional)	The options that will be used to set the metadata. The default value is None.	Dictionary

Return value

Data type	Explanation
Boolean	True if the metadata is set successfully; otherwise, False.

cloud_io = AIO(r"C:\data\datacloud.acs")
cloud_io.cloud.setmetadata(
    r"cog.tif",
    "HEADERS",
    {
        "x-amz-id-2": "HxM4BcALW+HnJ13owxzMpeQXXI59VKK3I9zujbqyLBCinCt6prnzKXR4Ixhb/AloFVJLGtuqKG7nSLCLaxObTw==",
        "x-amz-request-id": "KYTDPC2WH2ZQK2D2",
        "Date": "Tue, 13 Jun 2023 13:39:33 GMT",
        "Last-Modified": "Wed, 16 Sep 2020 13:49:34 GMT",
        "ETag": '"c220c099c46c60703de0c55763d04371"',
        "x-amz-meta-s3b-last-modified": "20171213T210220Z",
        "Content-Disposition": "attachment",
        "Accept-Ranges": "bytes",
        "Content-Range": "bytes 0-16383/4339079",
        "Content-Type": "text/plain",
        "Server": "AmazonS3",
        "Content-Length": "4545",
    },
)

`validate({certificate}, {stat}, {list}, {read}, {modify}, {path})`

Tests the cloud connection for various file operations. Available permissions may be granular depending on the server ACL configuration.

This method is only applicable for cloud.

Name	Explanation	Data type
certificate (Optional)	Specifies whether the SSL certificate and Online Certificate Status Protocol (OCSP) certificate check will be verified.	Boolean
stat (Optional)	Specifies whether stat on the root container or connected folder will be tested.	Boolean
list (Optional)	Specifies whether list directory access will be tested.	Boolean
read (Optional)	Specifies whether the read file permission will be tested.	Boolean
modify (Optional)	Specifies whether the write and delete permissions will be tested. If delete access is denied, a test file will remain on the cloud.	Boolean
path (Optional)	The path to the file that will be used for the read test. If no value is provided, the first entry from the current directory listing will be used.	String

Return value

Data type	Explanation
List	Results of the test and a detailed error message if available.

cloud_io = AIO(r"C:\data\datacloud.acs")
cloud_io.cloud.validate(True, True, True, True)