Read a Local File from a Folder
The Read a Local File from a Folder data source in ArcGIS Velocity can be used to read event data formatted as delimited text, JSON, or XML from a local file in a folder. You can find this data source listed under the Web and Messaging option.
Example
The following is an example use case for the data source:
A manufacturing company uses various sensors to monitor the health of its equipment. These sensors log data locally in delimited text and JSON formats, including temperature, vibration levels, and operational status. The logs are saved every minute to a folder on a local server. Using the Read a Local File from a Folder data source in Velocity, the company sets up a real-time analytic to monitor this folder.Velocity reads each new file as it appears, adapts the data, and checks for anomalies, such as temperature spikes or abnormal vibration patterns.
Usage notes
The following are the parameters for the data source:
Velocity supports multiple output data formats. These include JSON, GeoJSON, EsriJSON, and delimited text.
The data format can be detected automatically to define the data schema as you configure the data source.
When choosing to use the delimited text format, you must specify the delimiter to place between attribute values. Common delimiters are a comma, the pipe character, or a semicolon.
It is recommended that you use an absolute path for the File Directory parameter—for example,
C:\Velocity\test.You can use the File Filter parameter with options such as
*.json,*.txt,*.csv, or*.xmlto load multiple files from a folder. All files must have the same structure or schema; otherwise, Velocity shows an error when trying to read sample messages.Key fields such as the Location type, Date and time, and Track ID parameters are specified as part of source configuration.
If the file contains a control character outside the range of alphanumeric printable characters, or if there is a schema mismatch while the feed is running, Velocity throws an error.
To read files from a directory, the user account running Velocity must have read, write, and modify permissions on that directory. Without these permissions, Velocity fails to access or read files, resulting in errors.
If the Delete file after processing parameter is turned on, the file is deleted as soon as the data has been loaded into the real-time analytic and the analytic has been saved and started.
Parameters
The following are the parameters for the data source:
|
Parameter |
Description |
Data type |
|---|---|---|
|
File Directory (Required) |
The path to the folder containing the local files to be processed—for example, |
File path |
|
File Filter |
Specifies file within the file directory that should be included for processing. File filter can be applied based on file name and/or file type. The default is asterisk (*), which means all files in the directory are included. |
String |
|
Delete file after processing (Optional) |
Specifies whether the processed local files should be automatically deleted from the directory after successful processing. If this parameter is turned on, files are deleted from the directory after being processed. If this parameter is turned off, files are not deleted from the directory after being processed. Files that are not deleted are reread, from the beginning of the file, if the data source’s parameters are changed and saved or if the feed is stopped and restarted. |
Boolean |