Write GeoJSON to a Kafka Topic

The Write GeoJSON to a Kafka Topic output connector can be used to write event data, adapted and formatted as GeoJSON, to an Apache Kafka Topic. For more information about getting started with Apache Kafka, see Apache Kafka Introduction. For more information and examples of GeoJSON data record formatting, see the GeoJSON specification.

Usage notes

Keep the following in mind when working with the Write GeoJSON to a Kafka Topic output connector:

• Use this output connector to write data, adapted and formatted as GeoJSON, to a Kafka Topic. This output connector is a producer to Kafka.
• This output connector pairs the GeoJSON outbound adapter with the Kafka outbound transport.
• Like feature JSON, GeoJSON is a subtype of generic JSON. Different GeoEvent Server adapters are used to process these specific formats.
• This output connector supports writing GeoJSON to a Kafka Topic formatted as either generic GeoJSON or feature GeoJSON. Generic GeoJSON follows the GeoJSON standard, while feature GeoJSON is a custom format used with the spatiotemporal big data store.
• The Kafka Outbound Transport supports TLS 1.2 and SASL security protocols for authenticating with a Kafka cluster or broker.

Parameters

The following are the parameters for the Write GeoJSON to a Kafka Topic output connector:

Parameter	Description
Name	A descriptive name for the input connector used for reference in GeoEvent Manager.
Override with Custom Kafka Properties	Specifies whether the default GeoEvent Server Kafkaclient properties will be overridden. The default is No. Yes—The default Kafka client properties exposed by the transport are overridden. A folder registered with GeoEvent Server must be specified that contains a Kafka.properties file with the correct formatting for a valid Kafkaconfiguration. Refer to Apache Kafka Configuration for a list of supported configurations and expected formatting for the specified .properties file. No—The default Kafka client properties exposed by the transport are not overridden. Kafka Bootstrap Servers and Consumer Group ID values must be specified.
Kafka Bootstrap Servers (Conditional)	A list of hostname:port pairs used to establish the initial connection to the Kafka cluster. Hostname:port pairs must be comma separated, for example, broker0.example.com:9092,broker1.example.com:9092,broker2.example.com:9092.
Registered Folder for the Kafka Properties File (Conditional)	The folder registered with GeoEvent Server that contains the Kafka .properties file. The Kafka .properties file defines the custom Kafka properties when Override with Custom Kafka Properties is set to Yes. Ensure that the folder registered with GeoEvent Server is the full path to where the Kafka .properties file is located.
Kafka Properties File Name (Conditional)	The name of the Kafka .properties file that contains the custom Kafka properties for client configuration. Specify the name of the file without the .properties extension. If the name of the custom Kafka properties file is sample.properties, specify this parameter value as sample.
Topic Name	The name of the Kafka topic to publish data to. topic1 Note: The Kafka outbound transport does not support publishing data to multiple topics.
Enable Exactly Once Delivery	Specifies whether exactly once semantics are honored when writing to the topic. Refer to Exactly Once Semantics in the Apache Kafka documentation for more information. The default is Yes. Yes—Kafka honors Exactly Once Semantics. No—Kafka does not honor Exactly Once Semantics. Note: Ensure that the Kafka cluster supports exactly once semantics before continuing with the Enable Exactly Once Delivery parameter. Enabling exactly once semantics can affect performance. For more information, refer to Producer Configs as related to acknowledgments in the Apache Kafka documentation.
Formatted JSON	Specifies whether the feature JSON string is prettified with whitespace indentation and newline formatting. This JSON is only appropriate when writing to a system file whose content may be reviewed in a text editor. The default is No. Yes—The feature JSON is formatted (prettified). No—The feature JSON is not formatted.
Converter Type	Specifies the GeoJSON format of adapted event records. The GeoJSON format options are Generic GeoJSON and Feature GeoJSON. The default is Generic GeoJSON. Generic GeoJSON—Outbound event records are adapted using the standard GeoJSON specification. For more information about the standard GeoJSON format, refer to GeoJSON Specification. Feature GeoJSON—Outbound event records are adapted using a custom feature GeoJSON format; the format for adding data to the spatiotemporal big data store is used.
Ignore Reserved Field Names	Specifies whether reserved field names are ignored when adapting processed event data as GeoJSON for Kafka. Reserved field names include ObjectId, GlobalId, and UniqueId (field names are case sensitive). The default is No. Yes—Any reserved field names are ignored when adapting processed event data as GeoJSON. The reserved field names and their corresponding values are deleted from the schema. No—Reserved field names are not ignored when adapting processed event data as GeoJSON. Any reserved field names and their corresponding values are maintained as part of the data's schema.
Authentication Required	Specifies whether the connection to the Kafka cluster, or Kafka broker, requires authentication. The default is No. Yes—Authentication is required to connect to the Kafka cluster or broker. No—Authentication is not required to connect to the Kafka cluster or broker.
Authenticate Using (Conditional)	Specifies the security protocol that is used to secure the Kafka cluster. The security protocol options are TLS 1.2 and SASL. TLS 1.2—The security protocol used by the Kafka cluster is TLS 1.2. Ensure that the Kafka cluster's PKI file (x509 certificate) is imported into the trust store of the ArcGIS Server machine with which ArcGIS GeoEvent Server is configured. For details, refer to the Import the certificate into ArcGIS Server section in Configure ArcGIS Server with an existing CA-signed certificate for specific instructions on importing certificates. SASL—The security protocol used by the Kafka cluster is SASL. Only SASL, SSL, and Kerberos are supported. Note: When using Kerberos, ensure that the operating system user account running ArcGIS GeoEvent Server has read access to the keytab file in the Kerberos setup and configuration. This parameter is shown when Authentication Required is set to Yes.
Registered Folder for Credential File (Conditional)	The folder registered with GeoEvent Server that contains the Kafka cluster's PKI file (x509 certificate). Ensure that the folder registered with GeoEvent Server is the full path to where the Kafka cluster's certificate is located.
Credential Configuration File (Conditional)	The name of the Kafka cluster's PKI file (x509 certificate). The certificate and its associated private key must be stored in the PKCS#12 format, which is represented by a file with either the .p12 or .pfx extension. Provide the name of the file in addition to the extension. Note: Only the certificate file name and extension are supported for this parameter. Do not specify the relative paths to the certificate for this parameter. Register the full path to the certificate file using the Registered Folder for Credential File parameter. This parameter is shown when Authentication Required is set to Yes. It is applicable to TLS 1.2 only.
Keystore Password (Conditional)	The password for the Kafka cluster's PKI file (x509 certificate). This is also known as the certificate's private key.
SASL Authentication Type (Conditional)	Specifies the type of SASL authentication mechanism supported by the Kafka cluster. SASL authentication type options are SASL/GSSAPI (Kerberos) and SASL/PLAIN. SASL/GSSAPI (Kerberos)—The Kafka cluster uses SASL/GSSAPI Kerberos authentication. SASL/PLAIN—The Kafka cluster uses SASL/PLAIN authentication.
Kerberos Principal (Conditional)	The Kerberos principal for the specific user, for example, [e-mail].
Use Key Tab (Conditional)	Specifies whether the keytab in the Kerberos settings is used. The default is Yes. Yes—The keytab is used in the Kerberos settings. No—The keytab is not used in the Kerberos settings.
Store Key (Conditional)	Specifies whether the key is stored in the Kerberos settings. The default is Yes. Yes—The key is stored in the Kerberos settings. No—The key is not stored in the Kerberos settings.
Username (Conditional)	The username used to authenticate with the Kafka cluster. This is also known as a connection string with certain cloud providers. Refer to the documentation of the chosen cloud provider for correct syntax.
Password (Conditional)	The password used to authenticate with the Kafka cluster. Refer to the documentation of the chosen cloud provider for the correct syntax.

Considerations and limitations

The Write GeoJSON to a Kafka Topic output connector is a producer to Kafka. Apply the same considerations to this output connector as would be required for any other external producer to Kafka.