Change the location of Kafka and ZooKeeper data files

The ArcGIS GeoEvent Gateway uses a local folder for Apache Kafka on-disk topic queues and Apache ZooKeeper configuration files.

• On Windows, the default directory is C:\ProgramData\Esri\GeoEvent-Gateway.
• On Linux, the default directory is /home/arcgis/.esri/GeoEvent-Gateway.

When installing ArcGIS GeoEvent Server to a drive or file path other than the default, the ArcGIS GeoEvent Gateway uses the default folders above for the Kafka and ZooKeeper files. This can pose a problem if, for example, an organization’s standard practice is to limit the available disk on the C: drive and install software on another drive. If this is the case, you can reconfigure the ArcGIS GeoEvent Gateway to write data files to a folder other than one of the default folders. This change must be made after installing GeoEvent Server.

Follow the steps below, for either Windows or Linux, to change the location of the Kafka on-disk topic queues and ZooKeeper configuration files. It is recommended that you use the ArcGIS GeoEvent Manager web app to accomplish this.

Note:

• The steps below delete the existing GeoEvent Server configuration. It is recommended that you perform these changes immediately after installing GeoEvent Server and before configuring GeoEvent Server or deploying custom components.
• The system account used to run ArcGIS GeoEvent Server and the ArcGIS GeoEvent Gateway services on Windows or the equivalent daemons on Linux must have full control to read and write, create, and delete files to the file path you choose for the GeoEvent Gateway data files.
• Windows UNC path names are not supported. When specifying a gateway data directory, use a local server directory, such as C:\ProgramData\Esri or D:\Data\Esri on Windows, or a path under the user account used to run GeoEvent Server, such as /home/arcgis/.esri on Linux.

Export current configuration

If inputs, outputs, or other elements, such as GeoEvent Services or GeoEvent Definitions, are configured in GeoEvent Server, complete the following steps to export the current configuration before proceeding with the next sections:

1. Start ArcGIS GeoEvent Manager and click Site > GeoEvent > Configuration Store.
2. Click Export Configuration to create a backup of the existing configuration.

   If you have custom components (connectors or processors) deployed, it is recommended that you perform the next step to remove them from GeoEvent Server before proceeding with the next section.

3. With the ArcGIS GeoEvent Server service running, move existing *.jar files from the deploy folder to a temporary directory.

   The deploy folder is located at <ArcGIS Server installation directory>\GeoEvent\deploy.

Update the registered folder using GeoEvent Manager

After changing the directory location, update the registered folder path. Complete the following steps to update the registered folder in GeoEvent Manager:

1. Sign in to GeoEvent Manager.
2. Click Site > GeoEvent > Data Stores.
3. Under the Registered folders drop-down menu, locate the folder named Automatic Backups and review its status.

   If the Status value is red, it means the folder path has not been validated, as shown in the image below at label 1.

4. Click the edit button under the Edit column to open the Edit folder dialog box, as shown in the image below at label 2.
5. On the Edit folder dialog box, update the Path parameter to the correct location, as shown in the image below at label 3.

   For example, C:\ESRI\geoeventserver\GeoEvent. Ensure this path exists and is accessible by the GeoEvent Server service account.

6. Click Register to save the updated folder path.

   The screenshot below summarizes the steps to update the registered folder directly in the GeoEvent Manager web app.

   

7. Click Validate All to confirm the folder is valid.

Once the folder path is updated and validated, GeoEvent Manager recognizes the correct location for backups and data storage.

Change directory location using GeoEvent Manager for Windows and Linux

You can change the directory location in the GeoEvent Manager web app instead of editing configuration files manually on-disk. Complete the following steps to change the directory location in the GeoEvent Manager web app.

1. Sign in to GeoEvent Manager.
2. Click Site > GeoEvent > Directories, as shown in the images below at label 1.
3. Click Edit Directories to open the Edit Directories dialog box, as shown in the images below at label 2.
4. On the Edit Directories dialog box, do the following:
   1. Type the location for the Configuration Store option, as shown in the images below at label 3. See the following examples:

      For Windows: C:\ESRI\geoevent\GeoEvent

      For Linux: /home/arcgis/.esri/GeoEvent

   2. Type the location for the Gateway option. See the following examples:

      For Windows: C:\ESRI\geoeventserver\GeoEvent-Gateway

      For Linux: /home/arcgis/.esri/GeoEvent-Gateway

   3. Click Save.

Note:

GeoEvent Server may restart automatically to apply changes.

The screenshot shows the process to change directory location in the GeoEvent Manager web app for Windows.

The screenshot shows the process to change the directory location in the GeoEvent Manager web app for Linux.

Change the location on Windows (on-disk method)

You can change the directory location manually by editing configuration files on-disk.

1. Stop the ArcGIS GeoEvent Server and ArcGIS GeoEvent Gateway Windows services.
2. In File Explorer, browse to the ArcGIS GeoEvent Gateway configuration files in the etc folder located at the <ArcGIS Server installation directory>\GeoEvent\etc directory.
3. Open the com.esri.ges.persistence.file.cfg file in a text editor and update the gateway.data.dir property with the appropriate value.

   The default value for this property is as follows: gateway.data.dir=C:\\ProgramData\\Esri\\GeoEvent-Gateway.

   Note:

   The gateway data directory must specify an existing folder. The log directories specify a comma-separated list of subfolders that ArcGIS GeoEvent Gateway creates when the service is restarted.

4. Save and close the com.esri.ges.persistence.file.cfg file.
5. In File Explorer, browse to the ArcGIS GeoEvent Gateway configuration files in the etc folder located at the <ArcGIS Server installation directory>\GeoEvent\gateway\etc directory.
6. Open the kafka.properties file in a text editor and update the log.dirs property value with the appropriate value.

   The default value for this property is as follows: log.dirs=kafka/logs,kafka/logs1,kafka/logs2

   Note:

   ArcGIS GeoEvent Server 10.8.1 and earlier specified a single subfolder, kafka/logs. Starting with 10.9, multiple partitions and topics are configured requiring that three subfolders be specified (for example, kafka/logs,kafka/logs1,kafka/logs2).

7. Save and close the kafka.properties file.
8. In File Explorer, browse the ArcGIS GeoEvent Gateway configuration files in the etc folder located at the <ArcGIS Server installation directory>\GeoEvent\gateway\etc directory.
9. Open the zookeeper.properties file in a text editor and update the data.dir property value with the appropriate value.

   The default value for this property is as follows: data.dir=zookeeper-data

10. Save and close the zookeeper.properties file.
11. In File Explorer, browse and delete the files and folders in the data folder located at the <ArcGIS Server installation directory>\GeoEvent\data directory.
12. Restart the ArcGIS GeoEvent Gateway Windows service.
13. Wait for ArcGIS GeoEvent Gateway to start running and then start the ArcGIS GeoEvent Server Windows service.
14. Start ArcGIS GeoEvent Manager and confirm that you can sign in.
15. Optionally, if you deployed custom components, move the *.jar files back into the deploy folder located at the <ArcGIS Server installation directory>\GeoEvent\deploy directory.
16. Optionally, if you exported the GeoEvent Server configuration, import it using ArcGIS GeoEvent Manager by clicking Site > GeoEvent > Configuration Store and clicking Import Configuration.

Note:

If new files and folders are not created under the specified gateway data directory after the ArcGIS GeoEvent Gateway is restarted, it is likely a permission issue with the system account used to run ArcGIS GeoEvent Server and ArcGIS GeoEvent Gateway not having sufficient read and write file access to the specified folder.

Change the location on Linux (on-disk method)

You can change the directory location manually by editing configuration files on-disk.

1. Stop the ArcGIS GeoEvent Server and ArcGIS GeoEvent Gateway Linux daemons.
2. Browse to the ArcGIS GeoEvent Gateway configuration files in the etc folder located at the <ArcGIS Server installation directory>\GeoEvent\etc directory.
3. Open the com.esri.ges.persistence.file.cfg file in a text editor and update the gateway.data.dir property value with the appropriate value.

   The default value for this property is as follows: gateway.data.dir=/home/arcgis/.esri/GeoEvent-Gateway.

   Note:

   The gateway data directory must specify an existing folder. The data directory specifies a configuration subfolder that ArcGIS GeoEvent Gateway creates when the daemon is restarted.

4. Save and close the com.esri.ges.persistence.file.cfg file.
5. In File Explorer, browse to the ArcGIS GeoEvent Gateway configuration files in the etc folder located at <ArcGIS Server installation directory>\GeoEvent\gateway\etc.
6. Open the kafka.properties file in a text editor and update the logs.dir property value with the appropriate values.

   The default value for this property is as follows: log.dirs= kafka/logs,kafka/logs1,kafka/logs2

   Note:

   ArcGIS GeoEvent Server 10.8.1 and earlier specified a single subfolder, kafka/logs. Starting with 10.9, multiple partitions and topics are configured requiring that three subfolders be specified (for example, kafka/logs,kafka/logs1,kafka/logs2).

7. Save and close the kafka.properties file.
8. In File Explorer, browse to the ArcGIS GeoEvent Gateway configuration files in the etc folder located at the <ArcGIS Server installation directory>\GeoEvent\gateway\etc directory.
9. Open the zookeeper.properties file in a text editor and update the data.dir property value with the appropriate value.

   The default value for this property is as follows: data.dir=zookeeper-data

10. Save and close the zookeeper.properties file.
11. Browse and delete the files and folders located in the data folder located at the /home/arcgis/server > rm -rf GeoEvent/data/* directory.
12. Restart the ArcGIS GeoEvent Gateway daemon.
13. Wait for the ArcGIS GeoEvent Gateway to start running and then start the ArcGIS GeoEvent Server daemon.
14. Start ArcGIS GeoEvent Manager and confirm that you can sign in.
15. Optionally, if you deployed custom components, move the *.jar files back into the deploy folder located at the <ArcGIS Server installation directory>\GeoEvent\deploy directory.
16. Optionally, if you exported the GeoEvent Server configuration, import it using ArcGIS GeoEvent Manager by clicking Site > GeoEvent > Configuration Store and clicking Import Configuration.

Note:

If new files and folders are not created under the specified gateway data directory after the ArcGIS GeoEvent Gateway is restarted, it is likely a result of a permission issue with the system account used to run ArcGIS GeoEvent Server and ArcGIS GeoEvent Gateway being unable to read and write files to the specified folder.

Additional considerations

The Kafka on-disk topic queues and ZooKeeper configuration files must be organized in the same data directory. For example, on Windows, the default directory for the Kafka on-disk topic queues is C:\ProgramData\Esri\GeoEvent-Gateway\kafka\logs, and the ZooKeeper configuration files default directory is C:\ProgramData\Esri\GeoEvent-Gateway\zookeeper-data. Kafka and ZooKeeper are tightly coupled, and their runtime files must be collocated.

The ArcGIS GeoEvent Server and ArcGIS GeoEvent Gateway services are constantly communicating with one another. Data from event records flows through Kafka topics as it is collected, processed, and disseminated. The status of configured elements and information cached by GeoEvent Server is frequently updated in the ZooKeeper configuration store.

The ArcGIS GeoEvent Gateway runtime files must be stored locally on the server running the ArcGIS GeoEvent Gateway service on Windows or the equivalent daemon on Linux. It is recommended that you do not store the files on a network shared folder, a SAN, or a NAS. Storing the files in these locations can impact performance, solution reliability, and stability.