Real-time workflow considerations
When planning for a deployment of ArcGIS Velocity in an ArcGIS Enterprise environment, it is important to consider certain real-time workflow considerations for Velocity.
Real-time data processing
When Velocity receives data, five factors influence the time it takes for that data to be processed: size, rate, volume, analysis, and output destination.
Feature size
Real-time data can vary in the number of attribute values, the complexity of attribute values, and the geometry type (point, polyline, or polygon) in spatial contexts. String attributes may be a short name or may contain many characters. Points contain a pair of coordinate values, while a polyline and polygon contain a collection of coordinate-value pairs. Larger data (in bytes) requires more system resources to process and can take more time overall. Velocity uses CPU, RAM, and disk space to process real-time data as feature records. Larger or more complex feature records require more of these resources to process.
Feature rate
The load on Velocity is proportional to the rate at which features are ingested, adapted, and processed. Rate is the interval of time in which a volume of features is received. Larger volumes of features and fast arrival rates cause Velocity to draw more system resources (compute, memory, and disk space) to ingest and adapt raw sensor data, process the feature records it creates, and disseminate data from the processed feature records. For example, 5,000 sensors providing updates every 5 seconds represent a higher feature rate than 5,000 sensors providing updates every 10 minutes.
Feature volume
Feature volume measures the number of unique records in a system. Higher feature volumes increase the load on Velocity and require better CPU, RAM, and disk performance than lower volumes. The larger the feature volume, the higher the feature velocity for processing. For example, a large transportation department tracking the position of 15,000 state-owned vehicles represents a high feature volume. In contrast, a township tracking the location of 12 snowplows represents a low feature volume.
Feature velocity
The velocity of real-time data processed by ArcGIS Velocity is measured in features per second. Feature velocity (features per second) is determined by dividing the feature volume by the feature rate, expressed in seconds. A batch of 500 feature records arriving every 10 seconds equates to a feature velocity of 50 features per second.
Feature velocity can also be expressed in bytes. This involves taking the feature volume, expressed in bytes, and averaging it across each second. If a feature record is 20 bytes in size, and there are 500 feature records arriving every 10 seconds, this equates to a feature velocity of 1,000 bytes per second.
Whether expressed as features or bytes, velocity matters when considering the computational limits of a machine or the network on which it resides. Feature analysis can be affected when feature velocity exceeds the limitations of network speed, disk read-write operations, available compute, memory required to perform system operations, and more.
Feature analysis
As feature records pass through Velocity, the amount of processing and analysis performed on the features, using configurable tools, affects machine resources and feature processing time. Complex processing and analysis workflows typically use more CPU resources and increase the amount of time to process feature records.
It is recommended to minimize feature record volume and velocity in the analytics you configure while optimizing Velocity operations. For example, if data from network sensors is expected to arrive every second, you should not configure complex real-time analytics that take hundreds of milliseconds to run on each feature record. You should reduce the feature record volume, increase the feature rate, or simplify the filtering and processing that Velocity performs each second.
If data from a web service is expected to arrive every 30 seconds, Velocity can support complex real-time analytics if the volume of feature records received is completely processed before the next batch arrives.
Feature output destination
In many cases, the dissemination of feature data does not affect Velocity. However, certain considerations must be made when working with output destinations where the feature velocity exceeds the capabilities of a specific output destination. Different approaches may be required.
For example, attempting to write hundreds or thousands of records to a hosted feature service using a relational data store may result in failures. Instead, consider using a hosted feature service backed by a spatiotemporal big data store, which is designed for higher feature throughput and storage.
Learn more about types of data stores
A key capability of Velocity is the real-time processing of feature records against other feature records stored as geofences. Geofences can be established from a feed or source and are stored in memory. Therefore, the amount of machine RAM is important and determines how many geofences can be stored in memory. In addition to the amount of RAM, the type of RAM can affect the speed at which geofences are accessed, especially when a large number of geofences must be analyzed.
More important than the number of geofences is the complexity of those geofences. The more points, or vertices, the geofences have, the more memory ArcGIS Velocity requires to perform real-time spatial relationship analysis. Refer to the following examples of geofences—each may contain an equal number of points and vertices and may consume the same amount of machine RAM:
A set of 70,000 points representing points of interest across a region
A region’s road network containing several hundred road centerline segments of a few hundred points each
A state’s highly accurate boundaries for 25 counties, each containing several thousand points