Feature Layer (existing)

The Feature Layer (existing) output in ArcGIS Velocity allows you to write output features to an existing Velocity layer, an existing ArcGIS Online hosted feature layer, or an existing ArcGIS Server feature layer. When using this output type, features are written to an existing feature layer, as opposed to creating a feature layer. The output can accept multiple inbound pipelines. Each pipeline is validated against the destination feature layer schema and must carry that schema or a subset of it.

Examples

The following are example use cases for the output:

After ingesting real-time features from IoT sensors through a feed and a real-time analytic, an asset manager wants to add those observations to an existing feature layer.
A GIS analyst performs analysis of the latest information in a big data analytic and appends the results to an existing layer that is in use in various applications.
A business analyst configures a real-time analytic that ingests historical observations from an external cloud store, processes events as they occur, and captures the new observations in an existing feature layer.
An existing feature layer stores the latest state of each vehicle, with fields for location, engine status, and fuel level. Separate pipelines report each group of fields at different rates. Each pipeline connects to the same output and updates only its own fields for each track ID, and the other values are kept.

Usage notes

Note the following when working with the output:

The selected feature layer must be hosted by Velocity, be an ArcGIS Online hosted feature layer, or an ArcGIS Server feature layer.
The schema of the incoming data and the destination feature layer must generally match. Each must have the same geometry type and spatial reference, and the fields present in the incoming data must exist in the destination feature layer. Differences can include the following:
- The destination feature layer can have more fields not present in the incoming data that you can use for data enrichment.
- The TRACK_ID and START_TIME fields in the incoming data schema do not have to match those in the destination feature layer when using ArcGIS Online or ArcGIS Server feature layers.
- When multiple pipelines are connected, each inbound pipeline is compared with the destination schema. A pipeline with a field that does not exist in the destination feature layer returns a validation error.
Use the Map Fields tool to transform the schema of the incoming data in an analytic workflow to align with the schema of the output feature layer.
When the incoming data schema has a TRACK_ID value set, you can write to a feature layer (existing) that does not have a track ID if the destination feature layer is an ArcGIS Online or ArcGIS Server feature layer. Keep in mind that when you write with the Keep only latest feature for each Track ID value data storage method option, Velocity overwrites any existing TRACK_ID values in the target dataset with the most recent observation from the incoming dataset.
When the incoming data schema does not include a value for the TRACK_ID field, you can write to a feature layer (existing) output that does not have a track ID. Keep the following behavior in mind:
- When writing with the Keep only latest feature for each Track ID value data storage method option, Velocity writes all incoming features to the target dataset.
Reference ArcGIS Server feature layers by using an ArcGIS organization item ID. To do this, go to your ArcGIS organization content page and add an item that references the layer by URL. If applicable, store credentials with this ArcGIS organization item. You can then configure the hosted feature layer (spatiotemporal) existing output to point to this ArcGIS organization item ID.
For Velocity layers, a data retention policy is required for all output feature layers because data grows over time. When writing to an existing feature layer and using the Add all new features option in the Data storage method parameter, the selected feature layer must have a data retention policy configured.

Learn more about data retention
If the Data storage method parameter is set to Keep only latest feature for each Track ID value, Velocity updates the editor name, date, and time after the first observation.
When the Data storage method parameter is set to Keep only latest feature for each Track ID value, choose the Keep the newer feature for a Track ID based on Start Time option in the Advanced options section. Velocity then compares the time specified in the Start Time option of each incoming feature to the stored feature for that track ID and skips the write if the incoming feature is older. This preserves the latest observation when events arrive out of order (for example, from multiple feeds or analytics writing to the same layer).

This parameter requires a START_TIME field. If no START_TIME field is defined, each feature is treated as new. This applies only to spatiotemporal feature layers.
The Simplify feature geometry parameter only applies to Velocity-hosted spatiotemporal feature layers. If your feature geometries are already simple, you can leave this parameter off to improve write performance.

Multiple inbound pipelines

Multiple pipelines can connect to a single Feature Layer (existing) output in real-time and big data analytics. There is no upper limit on the number of inbound pipelines.

Each inbound pipeline is compared with the destination feature layer schema, which is already defined. A pipeline must match the destination schema or be a subset of it, and each field must have the same name and data type as the matching field in the destination. No pipeline needs to carry the full schema. Fields are not added to the existing feature layer. For example, if the destination schema is A, B, C, D, E:

Inbound pipeline	Pipeline schema	Result
Pipeline 1	A, B, C	Valid. A subset of the destination schema.
Pipeline 2	A, C, D, E	Valid. A subset of the destination schema.
Pipeline 3	A, B, C, D, E	Valid. Matches the destination schema.
Pipeline 4	A, B, F	Not valid. Field F does not exist in the destination feature layer, so validation returns an error.

Note the following when connecting multiple pipelines to the output:

All inbound pipelines must have the same geometry type as the destination feature layer.
Use the Map Fields tool on a pipeline to rename fields, remove fields, or change data types so that the pipeline schema aligns with the destination schema.
When the Data storage method parameter is set to Keep only latest feature for each Track ID value, each inbound pipeline must include the field tagged as Track ID so features from every pipeline can be matched to the correct record.
A feature layer output cannot be added more than once to the same analytic. To write several pipelines to one feature layer, connect each pipeline to the single output.

Field values are written as follows:

Fields that are not in a pipeline’s schema are not written for features from that pipeline. A field that is missing is different from a field that is present with a null value.
When the Data storage method parameter is set to Keep only latest feature for each Track ID value, a feature from a subset pipeline updates only the fields in that pipeline’s schema. Existing values in the other fields for that track ID are kept.
When the Data storage method parameter is set to Add all new features, each feature is added as a new record. Fields that are not in the pipeline’s schema are null for that record.
If a field is in a pipeline’s schema and its value is null, the null value is written and replaces the existing value.

Parameters

The following table describes the parameters for the output:

Parameter	Description	Data type
Feature Service Portal Item ID	The portal item ID of the corresponding feature layer item created in ArcGIS Online. The feature layer must be created by the user creating the analytic.	String
Data storage method	If checked (`true`), a spatiotemporal feature layer supports high-velocity ingestion, high-volume storage over time, dynamic aggregation, and automated data retention. This is the default. Add all new features—Velocity adds all new features to the feature layer. Keep only latest feature for each Track ID value—Velocity keeps only the latest feature for each unique track ID. The output data schema must have Track ID and Start Time options identified as key fields. When multiple pipelines are connected, only the fields in each pipeline’s schema are updated.	Boolean
Simplify feature geometry	Simplifies feature geometries to ensure that Velocity writes them to the database correctly. If you know the feature geometries are already simple, you can leave this parameter turned off to improve write performance. This parameter is available in the Geometry options section under Advanced options.	Boolean
Keep the newer feature for a Track ID based on Start Time	This parameter is available in the Advanced options section when the Data storage method parameter is set to Keep only latest feature for each Track ID value. When this parameter is set to Yes with the Track ID and Start Time options specified, Velocity keeps the newer feature per track ID and skips writes for older incoming features. When this parameter is set to No (default), incoming features overwrite the stored feature for a track ID as they arrive. This parameter requires a `START_TIME` field. If none is defined, a validation warning appears and each feature is treated as new.	Boolean
Creator and editor username	Specifies the creator or editor username for processed data. The username can be derived from a field of the incoming data or set as the analytic owner. The Creator and editor username parameter is available only when the existing feature layer has editor tracking turned on. Learn more about usage notes	String
Additional logging (optional)	Specifies whether Velocity enables logging for raw requests. The default is False. Note: Turn on this parameter only for troubleshooting, then turn it off when complete. Contact Esri Technical Support for help with troubleshooting.	Boolean

Considerations and limitations

Consider the following when using the output: