Requirements
If you have an ArcGIS Enterprise deployment, you can take advantage of a server-side asynchronous batch geocoding pattern that will help you get the best possible performance from your locator services. This approach works with locators that you have published on your portal and added as a utility service. For more information about creating locators, sharing them to a portal, and adding them as a utility service, see Configure the portal to geocode addresses.
Advantages
We can leverage the power of ArcGIS Enterprise to get the most out of your published geocoding services. Once a locator has been shared to your portal, it can be used to do batch geocoding. Some advantages to using your portal to do batch geocoding include:
- Maximize server resources to get the batch geocoding job done as fast as possible.
- Allow the ArcGIS administrator to manage back-end resources and not overload services.
- Perform large batch geocoding behind your organization's firewall using your data and your own published locator.
- Chunking and retry logic are built in to ensure large batch geocoding jobs do not fail.
Architecture
To get the most from your published locator services, you can scale up the number of instances of the geocode server. You can do this by going to the Services directory in ArcGIS Server Manager and, after selecting the geocoding service you want to scale up, go to the Pooling tab and change the Minimum number of instances per machine and Maximum number of instances per machine parameters. For geocoding, these numbers should always be equal. If you want four instances available at all times, you can change both these values to 4.
In the diagram above, the ArcGIS administrator has scaled up the locator service to four instances. The ArcGIS administrator for your organization can then set the numBatchThreads property to 4 for the locator, and achieve a fourfold increase in batch geocoding performance. The administrator can set the numBatchThreads property in the Portal for ArcGIS Directory sharing location. Note that if two users want to perform batch geocoding simultaneously, the ArcGIS administrator may want to set the numBatchThreads property to 2 or jobs will be queued and potentially time out while waiting for another job to finish. That way, each of batch geocoding job will be run against two of the available locator instances, and the back-end locator services will be able to handle the load of submitting multiple jobs simultaneously.
Configuration
The following steps describe how the ArcGIS administrator can configure their portal to leverage the asynchronous batch geocoding pattern described above.
Add logging for more feedback while batch geocoding
By default, the GeocodingTools geoprocessing service message level is set to Warning. For more detailed error messages, and for better updates on the status of your batch geocoding job, you can set the logging level of the GeocodingTools geoprocessing service to Info.
- Browse to the ArcGIS Server Manager of your hosting server.
- Click the Utilities folder in the Server Manager directory.
- Click the GeocodingTools geoprocessing service.
- Click the Parameters banner on the left and change Message Level to Info. Then, click the Save and Restart button in the upper right corner.
Changing the logging level to Info will cause a slight decrease in performance, but will provide better feedback of job progress and more information about possible errors, should they occur. These benefits can be seen in the Geocode Table, Geocode Locations From Table, and Geocode File tools in ArcGIS Pro.
Increasing the timeout for batch geocoding
By default, the GeocodingTools geoprocessing service times out on jobs longer than 200 minutes. If you anticipate a batch geoocoding job might take longer than 3 hours and 20 minutes to complete, you will want to increase the maximum time a client can use the service.
- Browse to the ArcGIS Server Manager of your hosting server.
- Click the Utilities folder in the Server Manager directory.
- Click the GeocodingTools geoprocessing service.
- Click the Pooling banner on the left and change the maximum time a client can use the service. Increasing this value to a very large number ensures that large batch geocoding jobs will never time out. Then, click the Save and Restart button in the upper right corner.
Configure the suggested number of concurrent batch geocoding requests (numBatchThreads) for your locator
The portal's locator can also be used by the GeocodingTools geoprocessing service running on your federated GIS Server site. The geoprocessing service can send multiple concurrent requests to the geocode utility service to speed up the overall execution for batch geocoding. The number of concurrent requests made by the geoprocessing service can be configured by a portal administrator. If the geoprocessing service is making excessive concurrent requests to the locator, it may perform poorly, and this could result in increased failures when running the geoprocessing service. For this reason, the administrator should set the number of instances used by the geoprocessing service based on the number of instances available for the locator service and also configure the number of concurrent batch geocoding requests that can be made from any instance of the GeocodingTools geoprocessing service.
As a best practice, the number of instances for the locator service must always be greater than or equal to the number of instances available for the GeocodingTools geoprocessing service. The number of concurrent batch geocoding requests should be set proportional to the number of instances available for both the locator and geoprocessing services. For example, if there are eight instances available for the locator service and two instances available for the geoprocessing service, set the concurrent batch geocoding requests to 4. When the geoprocessing service is executed, it sends four concurrent batch geocoding requests to the locator service. In case more instances are available for the locator service, adjust the number of concurrent batch geocoding requests appropriately to maximize the utilization of instances for the locator service.
Note that the Esri World Batch Geocoder cannot be configured to use more concurrent requests. The following configuration applies to all other published locator services that are configured as utility services.
To set the suggested number of concurrent batch geocoding requests for your locator service, follow these steps:
- Go to the Portal for ArcGIS Directory sharing location and sign in as a member with administrative privileges. The URL is in the format https://webadaptorhost.domain.com/<webadaptorname>/sharing/rest.
- Browse to Home > Portals > Self.
- At the bottom of the page, click Update.
- In the Geocode Service text box, you will see the locator services currently configured with the portal. For the services that allow
batch geocoding (those services that have the batchproperty set to true), you can configure the suggested number of
concurrent batch geocoding requests. Choose the service that you want to configure with this property. The JSON in the Geocode Service text box will be in the following format:
[{ "url" : "https://geocode.arcgis.com/arcgis/rest/services/World/GeocodeServer", "northLat" : "Ymax", "southLat" : "Ymin", "eastLon" : "Xmax", "westLon" : "Xmin", "name" : "Esri World Geocoder", "batch": false, "suggest" : true }, { "url" : "https://machine.domain.com/server/rest/services/Locators/USA/GeocodeServer", "name" : "Streetmap Premium USA Geocoder", "placeholder" : "Find address or place", "singleLineFieldName" : "SingleLine", "batch" : true, "placefinding" : true, "suggest" : true, "zoomScale" : 10000 }] - Add the property numBatchThreads to all of the locator services that you want to use with the GeocodingTools geoprocessing service. To set the numBatchThreads value, first determine the number of concurrent batch geocoding requests you want to enable for a given locator service, and set the property equal to that number. Note that to configure numBatchThreads on the locator service, the service must have batch set to true.
In this example, the number of allowable concurrent batch geocoding requests for the Streetmap Premium USA Geocoder utility service is set to 4.
[{ "url" : "https://geocode.arcgis.com/arcgis/rest/services/World/GeocodeServer", "northLat" : "Ymax", "southLat" : "Ymin", "eastLon" : "Xmax", "westLon" : "Xmin", "name" : "Esri World Geocoder", "batch": false, "suggest" : true }, { "url" : "https://machine.domain.com/server/rest/services/Locators/USA/GeocodeServer", "name" : "Streetmap Premium USA Geocoder", "placeholder" : "Find address or place", "singleLineFieldName" : "SingleLine", "batch" : true, "placefinding" : true, "suggest" : true, "zoomScale" : 10000, "numBatchThreads" : 4 }] - After you have updated the JSON in the Geocode Service text box, click Update Organization at the bottom of the page.
Tools enabled for batch geocoding
After the ArcGIS administrator has configured the numBatchThreads per locator, users can take advantage of the increased performance in their batch geocoding jobs. Tools that take advantage of this performance increase are Geocode Table, Geocode Locations From Table, and Geocode File in ArcGIS Pro, as well as Geocode Locations from Table in Map Viewer.