Skip to main content

Kafka

The Kafka output in ArcGIS Velocity can be used to send output features to a topic in an externally accessible Kafka broker. A Kafka message is sent for each output feature sent to the output.

Learn more about Apache Kafka

Examples

The following are example use cases for the output:

  • An analyst configures a real-time analytic to process and analyze events ingested by a feed and sends the resulting features to a Kafka topic.

  • A financial analyst sends real-time stock trade updates to a Kafka topic, allowing instant data access.

Usage note

Keep the following in mind when working with the output:

The following output data formats are available:

  • Delimited

  • JSON

  • GeoJSON

  • EsriJSON

Parameters

The following are the parameters for the output:

Parameter

Description

Data type

Brokers

A comma-delimited list of Kafka brokers, specified as server:port—for example, machine1.domain.com:9092,machine2.domain.com:9092,machine3.domain.com:9092.

String

Topic

The Kafka topic the output sends messages to.

String

Authentication

Specifies the type of authentication to access the HTTP endpoint. The following are the authentication options:

  • None

  • SASL/PLAIN

  • SASL/SCRAM-SHA-256

  • SASL/SCRAM-SHA-512

String

Use SSL

Specifies whether Velocity uses SSL (SASL_SSL) in attempting to connect to the Kafka broker.

Note:

This should match the configuration the broker has on a specified port. If you this turn parameter off, Velocity connects with PLAINTEXT.

Boolean

Username

Specifies the username for accessing the Kafka broker.

This parameter only applies when Authentication is set to SASL/PLAIN, SASL/SCRAM-SHA-256, or SASL/SCRAM-SHA-512.

String

Password

Specifies the password for accessing the Kafka broker.

This parameter only applies when Authentication is set to SASL/PLAIN, SASL/SCRAM-SHA-256, or SASL/SCRAM-SHA-512.

String

Output format name

The format for outgoing messages. The following are the supported output format names:

  • GeoJSON

  • EsriJSON

  • Delimited

  • JSON

String

Write header rows

Specifies if the files write out the header row with field names. The default is false.

Boolean

Field delimiter

The field delimiter used by the file to separate fields. The default is a comma.

String