Skip to main content

Amazon Simple Storage Service (S3)

The Amazon Simple Storage Service (S3) output in ArcGIS Velocity allows you to write output features to files in an Amazon S3 bucket.

Example

The following is an example use case for the output:

After performing analysis in a big data analytic, a researcher wants to write the output spatial or tabular features to files in an Amazon S3 bucket.

Usage notes

Keep the following in mind when working with the output:

  • The Dataset parameter specifies the name of the folder that Velocity writes to Amazon S3. This folder contains one or more files, depending on the size of the output.

  • The Secret key value is encrypted the first time the analytic is saved and it is stored in an encrypted state.

  • When specifying the Folder path value, use forward slash (/) symbols.

  • This output is only available in big data analytics.

  • The following output data formats are available:

    • Delimited

    • JSON

    • GeoJSON

    • EsriJSON

    • Parquet

    • Shapefile

  • Data exported in a delimited format includes the SHAPE and DATE fields. The shape field contains geometry in WKT format. The date field is provided if the data includes a START_TIME key field.

  • Data exported in the JSON format provides geometry in the EsriJSON format.

  • The following Amazon S3 actions are required for your user policy associated with the provided Amazon key for Velocity to successfully connect to your Amazon S3 bucket and data in the provided bucket and folder path to write data.

    • The s3:ListBucket action must be allowed for the specified bucket.

    • The following actions must be allowed on the specified folder path and subresources (arn:aws:s3:::yourBucketName/*) for an Amazon S3 output to write data:

      • s3:PutObject

      • s3:GetObject

      • s3:DeleteObject

Parameters

The following are the parameters for the output:

Parameter

Description

Data type

Access key

The Amazon access key ID for the S3 queue, for example, AKIAIOSFODNN7EXAMPLE.

Learn more about AWS security credentials

String

Secret key

The Amazon secret access key for the S3 bucket, for example, wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY.

Velocity uses the provided access key to load specified data sources into the application.

The secret key is encrypted the first time the analytic is saved and is stored in an encrypted state.

Learn more about AWS security credentials

String

S3 Bucket name

The name of the Amazon S3 bucket where the file or files are written.

String

Folder path

The folder path of the folder where the output dataset folder and files are written.

  • To write the output folder and files to the root level of an Amazon S3 bucket, use a single forward slash symbol, for example, /.

  • To write the output folder and files to a folder, use a forward slash symbol followed by the path to the folder, for example, /data_folder.

String

Dataset

The name of the folder that contains the output file or files. The file or files written to this folder are automatically named and divided into multiple files as needed. Do not include file extensions in the Dataset parameter.

String

Write mode

The write modes to write features to files in Amazon S3. The available write modes are as follows:

  • Overwrite

  • Append

  • Error if exists

String