There are several properties that can be configured to alter the settings for GeoAnalytics Server in your deployment. Configuring these settings for your organization is important to optimize the performance of GeoAnalytics Server. When configuring these settings, you must consider the following:
- The number of simultaneous jobs that will be running
- The size of the data being used
- How often jobs will be run
- The resources available
The big data analysis tools that portal members run are powered by tasks in the GeoAnalyticsTools system service in your ArcGIS Server site. The system properties below are set in the ArcGIS Server site, while the service properties and number of instances are set on the GeoAnalyticsTools system service itself. Before modifying any parameters, read through all of the setting options to plan your GeoAnalytics configuration.
The following examples will be used in this topic:
- Example 1: A three machine GeoAnalytics Server site that will run a single job at a time
- Example 2: A three machine GeoAnalytics Server site that will run two jobs simultaneously
Both of these examples include three machines (nodes), each with 16 GB of RAM and 4 CPU cores. In total, each GeoAnalytics Server site has 48 GB of RAM and 12 cores.
Note:
It is recommended that you not allocate more than 30 GB of memory per machine to GeoAnalytics.
System properties
The following system properties are optional properties that can be configured by a site administrator to allocate CPU and memory for the compute platform worker process on each machine in the ArcGIS Server site:
- percentageMaxAllowedComputeCores—This represents the maximum CPU cores (as a percentage of total available on a machine) that will be used by the compute platform worker process on a single node. The default and maximum allowed value is 80 percent. If the specified value translates to less than 1 CPU core, then 1 CPU core will be used for the compute platform worker process.
- percentageMaxAllowedComputeMemory—This represents the maximum memory (as a percentage of total available on a machine) that will be used by the compute platform worker process on single node. The default and maximum allowed value is 80 percent.
To edit the system properties, complete the following steps:
- Log in to the ArcGIS Server Administrator Directory as a user with administrative privileges. The URL is in the format http://gisserver.domain.com:6080/arcgis/admin/.
- Go to System > Properties.
- Click Update.
- Add the properties in the following JSON format: { "percentageMaxAllowedComputeCores": <value>, "percentageMaxAllowedComputeMemory": <value>}. For example: {"percentageMaxAllowedComputeCores": 80, "percentageMaxAllowedComputeMemory": 80}
Service properties
There are two service level properties in the GeoAnalyticsTools service for allocating maximum allowed memory and CPU cores per job. The default is set to 6 GB per machine and 4 CPU cores. It is recommended that you update these to an appropriate value based on the memory and CPU resources available in your site. Using the default will not take advantage of distributed compute and will result in poor performance.
To modify these properties, you can do the following:
- Log in to ArcGIS Server Manager as a user with administrative privileges. The URL is in the format https://gisserver.domain.com:6443/arcgis/manager.
- On the Services tab, select the System folder.
- In the list of available services, locate and choose to edit the GeoAnalyticsTools service.
- Use the Parameters tab to set the properties for the service. These include the following:
- Maximum allowed memory per job per machine (GB)—This represents the amount of memory in gigabytes (GB) that can be used per job per machine.
- Maximum allowed compute cores per job (CPU)—This represents the total number of CPU cores that can be used per job across all machines in your cluster.
- Once you are finished, click Save and Restart to update the service properties.
If the initial values are not modified, a GeoAnalytics task will only use only a part of the resources available (4 cores and 18 GB of the available 12 cores and 48 GB for the example setup).
For Example 1, you want to use up to 80 percent of the available compute cores and compute memory (as described in System properties). With a site total of 12 CPU cores and 48 GB RAM, you can calculate 80 percent of the memory per node as 0.80 x 16 = 12.8, and the 80 percent of the compute cores per job as 0.80 x 12 = 9.6 GB.
To take advantage of the resources available in the GeoAnalytics Server and to distribute the task across multiple machines of the site, you can update the values to the following:
- Maximum allowed memory per job per machine (GB): 12
- Maximum allowed compute cores per job (CPU): 9
This will ensure that the job will be distributed to more than two machines in the site and will use just under 80 percent of the available resources across all machines. The allocation set will also impact the number of instances as described below.
For Example 2, you want to use up to 80 percent of the available compute core and compute memory (as described in System properties) and be able to run two jobs at once. To do this, set each individual job to half of what you set above. This means the resources are available for two jobs at all times. With a site total of 12 CPU cores and 48 GB RAM, you can calculate 40 percent of the memory per node as 0.40 x 16 = 6.4, and the 40 percent of the compute cores per job as 0.40 x 12 = 4.8 GB.
To take advantage of the resources available in the GeoAnalytics Server and to distribute the task across multiple machines of the site, you can update the values to the following:
- Maximum allowed memory per job per machine (GB): 6
- Maximum allowed compute cores per job (CPU): 4
Note:
When selecting the amount of memory to use, be sure to set a number that is lower or equal than the percentage set for machine resources (default 80 percent). If you do, jobs will wait for resources that are not actually available for one minute, and then be canceled with the following error: ERROR BD_101057: Unable to start distributed job. Please check your GeoAnalyticsTools service settings and ensure that there are enough resources available for the job to run.. You will also see this error if resources are unavailable because other jobs are using the resources.
Minimum and maximum instances of the GeoAnalyticsTools service
The default minimum and maximum instances of the GeoAnalyticsTools service is set to one per ArcGIS Server machine in the site. Depending on the number of machines (n) that are in the GeoAnalytics Server site, there will be (n) number of instances available for accepting GeoAnalytics jobs. Depending on the number of instances and amount of memory and CPU cores allocated per job in the service properties, one or more concurrent requests may be processed. Any additional requests to the GeoAnalytics Tools will be queued until the instance or memory and CPU resources become available. If no resources become available after 1 minute, the job will be cancelled.
Allocating the maximum possible amount of memory and CPU cores per job is ideal for jobs that require it.
To process multiple jobs at the same time, you can increase the maximum instance as needed to more than one per machine in the site. However, you will also have to modify the per job CPU and memory limits so that there will be enough CPU and memory resources available for each concurrent job.
To set the minimum and maximum instances of the GeoAnalyticsTools service, complete the following steps:
- Log in to the ArcGIS Server Administrator Directory. The URL is in the format https://gisserver.domain.com:6443/arcgis/admin.
- Go to Services > System > GeoAnalyticsTools.
- Click Edit.
- Edit the maxInstancesPerNode and minInstancesPerNode properties and click Save Edits.
In the 3-machine example, there will be 3 total instances available sufficient to satisfy both the Example 1 and Example 2 use cases. To run more than 3 concurrent jobs with this 3 machine scenario, in addition to adjusting the CPU and memory limits, you must also update the number of instances.
The total CPU and memory limits across all instances should be less than the system limit for CPU and memory. The total memory per node multiplied by the number of instances must be less than what's available on a single machine (that is, (16 GB x 0.80) / 2 instances = 6.4, and must be an integer -> 6), and the compute cores must be less than the total cores divided by the number of instances (that is, (12 cores x .80) / 2 instances = 4.8 -> 4).
Timeout of the GeoAnalyticsTools service
By default, any long-running job processed by the GeoAnalyticsTools service will time out after 24 hours. You may want to edit this property so that tools will time out after running for longer or shorter periods of time. You can edit the timeout property for the GeoAnalyticsTools service in ArcGIS Server Manager following the steps below.
- Log in to ArcGIS Server Manager. The URL is in the format https://gisserver.domain.com:6443/arcgis/manager.
- On the Services tab, choose the System folder.
- In the list of available services, locate and choose to edit the GeoAnalyticsTools service.
- Use the Pooling tab to set the timeout property for the service under the Specify service timeouts section. Change The maximum time a client can use a service to the time in seconds that you want for the new timeout property.
- Click Save and Restart to update the service.