Skip To Content

Cache creation in Amazon EC2

Available with Standard or Advanced license.

Creating an ArcGIS map or image service cache on the Amazon Elastic Compute Cloud (EC2) instances differs from caching outside the cloud in the following ways:

  • You have a number of different instance sizes and prices at your disposal.
  • You can add volumes to the EC2 instance where you can place the cache.

The following sections discuss the above factors in more detail.

Choosing an instance size and price

Amazon EC2 offers a variety of instance sizes and specifications. Each has its own price per hour of usage. The larger instances, especially those with a lot of memory, can generate tiles quickly. The smaller instances generate tiles more slowly but have a lower cost.

You can create your cache on an attached Amazon Elastic Block Store (EBS) volume using a powerful instance. When the caching completes, you can detach the EBS volume and attach it to your regular instance (which may be smaller and less expensive). You can then terminate the powerful instance that you used for caching. In this way, you can cache in the cloud while not committing to a relatively expensive instance for any longer than necessary.

You must decide between economy and speed. Using a low power instance with a low cost per hour is not always the most economical choice, as the total cost of the cache is dependent on the number of hours spent creating tiles. On the other hand, the most powerful instances may also yield a higher total cost of the cache—even though you spend fewer hours caching, you pay a higher price per hour.

Using a small test cache (perhaps the size of a medium-sized city) as well as a custom Amazon Machine Image (AMI) or site template, you can perform relatively inexpensive tests with different instance types to find out which is most economical for your cache.

Powerful EC2 instance types are well suited to scheduled cache updates, because many update workflows are time sensitive.

Choosing the number of map service instances to use when caching

Each EC2 instance has a certain number of virtual CPU cores. This number is visible when you choose the instance type in the Amazon Web Services (AWS) Management Console. The number of cores can help you determine how many instances of the CachingTools geoprocessing service to devote toward your caching. Using too many service instances will overwork your CPUs, while too few service instances will leave your CPUs underutilized.

Although the best number may be reached with some trial and error, a good starting point is to allow a maximum of n + 1 instances of the CachingTools service, where n is the number of virtual cores on a single EC2 instance in your site.

Automatic scaling

When building a large cache, you may be tempted to set up EC2 Auto Scaling triggers that increase the number of EC2 instances working on the cache as the CPU usage increases. However, Auto Scaling is better suited to handling unexpected spikes in traffic. Rather than use Auto Scaling, launch all the EC2 instances you'll need before you build the cache. This avoids having to wait for instances to launch sequentially, as will happen when you use Auto Scaling.

Deciding where to place the cache

As described in Strategies for data transfer to Amazon Web Services, there are several types of locations where you can place your data. When you create the cache, you can write it to an EBS volume that's attached to your EC2 instance or you can write it to an Amazon Simple Storage Service (S3) bucket.

If you create cache on the EBS volume, ensure the volume is large enough. If the volume is too small, you can replace it with a larger volume you create from a snapshot of the existing data volume and register a server cache directory on it. Do not build a cache on the C or root drive of the EC2 instance. If the instance is ever terminated, the cache will be lost.

To store map and image caches in an S3 bucket, follow these steps:

  1. Create an Amazon S3 bucket in the same region as your ArcGIS Server site on AWS.
  2. Create a folder in the S3 bucket. Name the folder arcgiscache.
  3. Sign in to ArcGIS Server Manager of the site on Amazon Elastic Compute Cloud (EC2) to which you want to publish cached map or image services and register the S3 bucket as a cloud store and a cache directory.

    You can use an AWS Identity and Access Management (IAM) role or the Access Key for credentials when you register the cloud store.

    Do not include the folder name, arcgiscache, in the S3 Bucket Name or Folder fields when you register the cloud store.

    Ensure the ArcGIS Server account has read and write access to the bucket using the Access Key or IAM role account you provide.

After you register the cloud store, you can create cache schema, place it in the S3 bucket, and publish cached map or image services using the cache in the S3 bucket. You can also update cache tiles and delete cache tiles in S3.

Considerations for caching in S3

The following are not supported if your cache is in S3: