When generating caches for raster tiles generated using map image layers and imagery layers, there are a number of tools and processes you need to be familiar with.
Note:
The term map service can be used interchangeably with the term map image layer, and the term image service can be used interchangeably with imagery layers.
Caching toolset
Cache management tools for server services like cached map layers, image layers, and tile layers are located in the Server Tools > caching toolset in ArcGIS Pro. See An overview of the Caching toolset for more information.
Note:
All caching tools require publisher or administrator credentials to use.
Server resources for caching
Generating and managing caches in ArcGIS Server is supported by a preconfigured caching controller and caching tool geoprocessing services. These system services are readily available when you create your ArcGIS Server site. The number of instances you allow for the CachingTools service determines how much power your machine can dedicate to caching jobs, while the number of Caching controller service instances in your System folder determines how many jobs can run at the same time.
Increasing the number of instances of a map or image service will not affect how fast tiles are created. The map or image service instances are used to fulfill the query and identify operations on the service and are not used for rendering the cached content. You can decrease the load on your server resources by taking advantage of shared instances for cached map services by configuring pooling of your map and image services. See Configure service instance settings for information.
Choose the number of instances for the CachingTools service
You can use ArcGIS Server Manager at any time to adjust the maximum number of instances of the CachingTools geoprocessing service that you want to make available for caching jobs. The minimum and maximum values apply to each individual GIS server, and as such, if your maximum is set to a value of 3 and you have four GIS servers running the CachingTools service, you could have up to 12 instances of CachingTools running.
This behavior allows you to add and remove GIS servers from your site to increase or decrease the number of resources dedicated to caching. You can add a GIS server even when a caching job is running, and it will be detected and assigned tiles to be created.
It is important that you choose an appropriate amount of instances of the CachingTools service. If you allow too many instances, your machine can become overwhelmed and inefficient. However, if you allow too few, your machine can go underutilized. Finding the best number for your circumstances can be a process of trial and error. However, it is recommended that you start by allowing a maximum of n, where n is the number of CPU cores on a single machine in your cluster.
Choose the number of instances for a caching job
Tools such as Manage Map Server Cache Tiles allow you to choose how many instances of CachingTools will work on the job. You can choose to divide the available instances of CachingTools among several running jobs. A job might not utilize its maximum number of instances of CachingTools if those instances are being used by other jobs. If a caching job is using all the CachingTools instances, other requested jobs are queued until the first job finishes.
Suppose you want to create a cache and you have four GIS servers in a site. You've configured each server to allow a maximum of five instances of CachingTools. The maximum number of instances you can dedicate toward any caching job is 20.
If you want to run two simultaneous caching jobs on this site and maintain an evenly distributed load, you can dedicate 10 instances toward each job.
Allowing for elasticity
It may be that you have configured your site in a cloud environment that can automatically add GIS servers in response to demand. In this case, you might not want to be limited by a fixed maximum number of instances that can work on the job. In this situation, you can enter a value of -1 to indicate that there is no limit on the number of instances that can work on the job. All the available instances of CachingTools will be used for the job, no matter how many GIS servers are added to your site.
Setting the number of jobs that can run simultaneously
If too many publishers start requesting cache to be built at the same time, the server can get overwhelmed, even if you only choose to dedicate a small number of instances toward each job. The CachingControllers service (in the System folder) determines how many jobs can run at the same time.
The default maximum number of instances for the CachingControllers service is 3, meaning only three caching jobs can run at once. If the server receives a request for a fourth caching job, it will be queued until one of the other jobs has finished. If you want to allow four jobs to run at once, you can set the maximum number of instances of CachingControllers to 4.
Automate cache creation and updates with geoprocessing
If you're working with a basemap that is unlikely to change, chances are you'll create its cache only once. You can launch ArcGIS geoprocessing tools to create the cache and add tiles to it. This is also an appropriate way to update the cache if your data does not change very often.
If your data changes often, you still might be able to use caching. The caching tools have been designed to allow you to script cache updates and target specific parts of the cache for updates. For example, you can do the following:
- Update the cache only for scale levels that have changed.
- Update only areas of the cache that fall within the boundaries of a feature class.
- Automate cache maintenance with utilities such as Windows Task Scheduler.
The way to automate the caching process is to write scripts that use tools from the Caching toolset in the Server Tools toolbox.
To create a new map or image service cache, you will typically start with Create Map Server Cache tool, which initializes the cache. Then you run Manage Map Server Cache Tiles to add tiles to the cache.
There are many tools that work with exporting, importing, and deleting caches. For a summary of all available tools, see An overview of the Caching toolset.
Using the tools in scripts
Although you can manually open the caching tools from the Catalog or Search window, it's more efficient to create a model in ModelBuilder or write a geoprocessing script that contains one or more tools that you want to run. You can then schedule scripts to run automatically.
The geoprocessing tool reference topic contains a full Python code sample showing how to use the tool in a script. For example, see the help for Manage Map Server Cache Tiles.
If you're new to Python, these resources can help you learn more:
- What is Python?—The ArcGIS Help contains an introduction to Python scripting, starting with this topic.
- Exporting a model—If you've created models in ModelBuilder, you can export them to Python to see what the corresponding script looks like.
Scheduling your script to run on a regular basis
You can save your script as a Python script file (.py). Double-clicking the script in Windows Explorer will cause the script to run. This is useful for testing, but in most cases, you'll want to schedule the script to run automatically on a regular basis.
Your operating system contains utilities that help you schedule tasks such as running a script. In Windows you can use Task Scheduler or the schtasks command. You'll need to provide the location of the script file, how often you want it to run, and the name and password that the task will run as. For further reading, see Scheduling a Python script to run at prescribed times.