Configure and deploy ArcGIS Enterprise for raster analytics

Raster analytics using ArcGIS Enterprise is a flexible raster processing, storage, and sharing system that employs distributed computing and storage technology based on ArcGIS Image Server. Use raster analytics to apply raster processing tools and functions offered in ArcGIS, build your own custom functions and tools, or combine multiple tools and functions into raster processing chains to execute your custom algorithms on large collections of raster data. Source data and processed results are stored, published, and shared across your enterprise according to your needs and priorities.

This extensive capability can be further expanded by leveraging cloud computing capabilities and resources. The net result is that image processing and analysis jobs that used to take days or weeks can now be done in minutes or hours, and jobs that were too large or daunting can now be handled.

Introduction

Raster analytics is ArcGIS Image Server configured for raster analysis in a processing and storage environment that maximizes processing speed and efficiency. Built-in tools and functions cover preprocessing, orthorectification and mosaicking, remote sensing analysis, and an extensive range of math and trigonometry operators. Your custom functions can extend the platform's analytical capabilities even further.

Fully utilize your existing ArcGIS Image Server on-site, or exploit the elastic processing and storage capacity of cloud computing and storage platforms such as ArcGIS Enterprise on Amazon Web Services and ArcGIS Enterprise on Microsoft Azure to dynamically increase or reduce your capacity depending on the size and urgency of your projects. The scalable environment of raster analytics empowers you to implement computationally intensive image processing that used to be out of reach or cost-prohibitive. This implementation saves you time, money, and resources.

Raster analytics is also designed to streamline and simplify collaboration and sharing. Users across your enterprise can contribute data, processing models, and expertise to your imagery project, and share results with individuals, departments, and organizations in your enterprise.

Configure a raster analytics deployment

The foundation of raster analytics is composed of 3 main components: ArcGIS Enterprise; raster analysis, based on ArcGIS Image Server; and an image hosting server, including raster data store, based on ArcGIS Image Server.

Raster analysis requires ArcGIS Enterprise, which provides software and user interfaces to organize and manage your processing, storage, and sharing of raster and feature data, maps, and other geographic information on a variety of devices. This integrated system manages the dissemination of processing and storage of results on-premises and behind the firewall, in cloud processing and storage environments, or a combination of both environments.

It includes Enterprise, ArcGIS Server, ArcGIS Data Store, and ArcGIS Web Adaptor. See the tutorial to set up a base ArcGIS Enterprise deployment.

The ideal deployment of raster analytics is composed of three server sites to perform the primary roles of the portal host server, raster analysis server, and image hosting server. Two licenses are required for raster analytics, ArcGIS Enterprise and ArcGIS Image Server.

The advantage of configuring raster analysis with three servers having distinct roles is to maximize efficiency and productivity. Separating distributed raster analysis processing and image service hosting roles enables resources to be dedicated to specific tasks without interruption. In this way, multiple users requesting access to various raster products and services do not compete for and impact raster analysis and image processing resources.

ArcGIS Enterprise

ArcGIS Enterprise integrates the components of the raster analytics system to support scalable raster processing and storage workflows.

The hosting server is your portal's server for standard portal administration and operations such as managing and dispensing processing, storage, and publication of results to raster analysis servers, image servers, and data stores. It also hosts the ArcGIS Data Store for GIS data and allows users to publish data and maps to a wider audience as web services. The following four components are required for the base ArcGIS Enterprise deployment:

• Portal for ArcGIS—First you will need to install Portal for ArcGIS, which is used to communicate with the raster analysis clients such as ArcGIS Pro or Map Viewer. These clients provide the interface to set up and process specific raster analysis functions, models, and function chains. Portal for ArcGIS allows you to publish and share maps, applications, and other geographic information with members of your organization. See Getting started with Portal for ArcGIS for information on installing and configuring Portal for ArcGIS to meet your organization’s requirements.
• An ArcGIS Server site licensed as the GIS Server and set as your portal’s hosting server. Designating an ArcGIS Server site as a hosting server requires a Server Enterprise Standard or Server Enterprise Advanced license.
• Two installations of ArcGIS Web Adaptor, where one installation is configured with the portal and one is configured with ArcGIS Server.
• One installation of ArcGIS Data Store, configured as a relational data store and set as your portal's managed database.

Your portal needs to be integrated with a federated hosting server that allows users to publish data and maps to a wider audience as web services. The portal administrator enables this by designating an ArcGIS Server site to perform as a hosting server. The hosting sever needs to be configured with ArcGIS Data Store set up with the relational database. ArcGIS Data Store is required because some of the raster analysis services will generate feature service output that is stored in the relational database and hosted on the hosting server. For a synopsis on setting up ArcGIS Server as a hosting server, see About using your server with Portal for ArcGIS.

Generally, a single-machine ArcGIS Server deployment is standard if you work with hundreds or a few thousand scenes and a few dozen GIS layers for a project. If you work in a very demanding environment supporting many simultaneous users, or process very large collections of imagery and large volumes of GIS layers, you should consider a multiple-machine ArcGIS Server deployment.

Esri provides templates to enable you to deploy ArcGIS Enterprise on-premises or in the cloud. These templates set up the portal, GIS hosting server, and data store for your configuration of choice. For a single-machine deployment, use the following templates:

• Single-machine deployment on-premises—See ArcGIS Enterprise Builder.
• Single-machine deployment using CloudFormation templates on Amazon Web Services.

For a multiple-machine deployment, use the following templates:

• Multiple-machine deployment on-premises—See the highly available system configuration template.
• Provision a highly available ArcGIS Enterprise deployment on Amazon Web Services.

Raster analysis

Distributed raster processing is performed by the ArcGIS Image Server assigned to the role of raster analysis. It executes the raster function processing chains on any supported raster data types. Source imagery and raster data can come from the distributed raster data store, a local client such as ArcGIS Pro, or other servers not configured in your raster analysis deployment. Ideally, your source data is staged on the distributed raster data store for optimum performance and efficiency.

Members of your portal will not interact directly with ArcGIS Image Server; they will use the ArcGIS Pro or web Map Viewer clients, ArcGIS REST API, ArcGIS Python API, or JS APIs to perform raster analysis. ArcGIS Image Server will perform the actual processing, storage, and publishing roles.

You need to set up and federate an ArcGIS Image Server in the role of a raster analysis server to perform distributed raster analysis and processing workflows. The raster analysis server automatically manages and distributes image processing and raster analysis tasks at the block level or the scene level depending on the tools and functions used. The raster analysis server manages the distribution of processing results to either ArcGIS Data Store on the hosting server for feature data products, or to the raster data store for imagery and raster data products.

The steps to set up and deploy an on-premises ArcGIS Image Server for raster analysis are outlined below.

Set up an ArcGIS raster analysis server

The following instructions may require changes to the way you've deployed ArcGIS in your organization; review them carefully before proceeding.

1. Make sure that you have configured a base ArcGIS Enterprise deployment.
2. Install an ArcGIS Server site to function as the ArcGIS Image Server assigned to the role of raster analysis server.
3. Authorize ArcGIS Server with an ArcGIS Image Server license.
4. Federate your ArcGIS Server site with your portal.
5. Register the data store with ArcGIS Image Server. Raster analysis services store processed rasters in a data store. This can be either a file share or cloud store type of data store.
   1. To use a folder, use ArcGIS Server Manager, register a UNC share in the data store, and name it _raster_store.

      Note:

      If there are multiple instances of _raster_store defined, for example, _raster_store1, _raster_store2, and so on, raster analysis will randomly pick one for a task request.

   2. To use a cloud store, you need to provide the access key. The cloud store settings include subfolder support.
6. Assign your federated ArcGIS Image Server to perform as your portal's raster analysis server.
   1. Sign in to the portal website as an administrator and go to Organization > Edit Settings > Servers.
   2. From the Raster Analysis section of the page, click the drop-down list and choose the ArcGIS Image Server site that you federated with the portal. If you added multiple ArcGIS Image Server sites to your portal, only one can be designated as the raster analysis server.
   3. Click Save.

      The settings page checks whether the server has an ArcGIS Image Server license.

7. Assign the minimum permissions needed to perform raster analysis.
   1. While still signed in to the portal website as an administrator, go to Organization > Role.
   2. Beside the role that will be performing raster analysis, click Edit.
   3. Check the boxes for the minimum privileges for publishing under Content, which are Create, update, and delete and Publish hosted feature layers.

      

   4. Check the Raster Analysis privilege box under Content and Analysis.

      

8. It is recommended that you configure raster analysis services.
ArcGIS Image Server raster analysis services have a default processing capacity configuration. You may need to customize this configuration based on your hardware configuration. Log in to ArcGIS Server Manager and browse to Services > System folder, where you can configure the following GIS services:
• To increase the processing speed and capacity, increase the maximum instance of the RasterProcessing service.
• To increase availability of the analytical results, increase the maximum instance of the RasterRendering service.
• To increase the number of processing tasks that can be executed concurrently, increase the maximum instance of the RasterAnalysisTools service.
9. Increase the SOC maximum heap size.
   1. Log in to the ArcGIS Server Administrator Directory using the URL format http://machine.domain.com:6443/arcgis/admin>.
   2. From the directory home page, click machines.
   3. From the list of machines, select the first machine licensed with ArcGIS Image Server, whether it is used as an image hosting server or as a raster analytics server.
   4. In the Server Machine Properties menu, check the value for SOC maximum heap size. If it is set to 64MB, click Edit in the Supported Operations list.
   5. On the Edit Machine page that opens, change the value for SOC maximum heap size from 64MB to 128MB. Click Save Edits. The server restarts.
   6. Repeat these steps for each machine in your deployment that is licensed with ArcGIS Image Server.

Note:

If you want to deploy your image hosting server in the cloud with Amazon Web Services, see ArcGIS Enterprise Cloud Builder CLI for AWSand AWS CloudFormation and ArcGIS. If you want to deploy your raster analysis server in the cloud with Microsoft Azure, see Deploy ArcGIS Enterprise on Microsoft Azure.

Image hosting server

The image hosting server hosts the raster data store and manages the file share storage as well as the cloud storage, and distributes results according to portal member requests. The raster data store is not part of ArcGIS Data Store but rather a registered file share or cloud share store federated with ArcGIS Enterprise. The raster data store is used by the raster analysis services to store output imagery optimized for reading, writing, and storage in a distributed format on-premises or in the cloud.

The advantage of configuring raster analysis with three servers having distinct roles is to maximize efficiency and productivity. Separating distributed raster analysis processing and image service hosting roles enables resources to be dedicated to specific tasks without interruption. In this way, multiple users requesting access to various raster products and services do not compete for and impact raster analysis and image processing resources.

You need to set up and federate a second ArcGIS Image Server to function as the image hosting server, which hosts all the distributed image services generated by the raster analysis server. It includes the raster data store configured with ArcGIS Image Server Manager, which manages distributed file share storage and cloud storage such as Amazon Simple Storage Service (S3) or Microsoft Azure Blob storage of image services. The image hosting server stores and returns distributed results requested by members in the Enterprise portal. The image hosting server requires an ArcGIS Image Server license.

The instructions to set up and configure the image hosting server are detailed below.

The following instructions may require changes to the way you've deployed ArcGIS in your organization; review them carefully before proceeding. To configure the portal to perform image hosting services, follow the steps below.

1. Install ArcGIS Server.
2. Authorize ArcGIS Server with an ArcGIS Image Server license.
3. Log in to ArcGIS Server Manager and browse to Site > Data Stores. Register a new raster store.

   The raster store is a location for output from raster analysis and should not be used as an input data directory. File shares can be registered directly as a raster data store. For cloud-based stores, first register a cloud store. Then create a raster store to use this cloud store.

   Note:

   If there are multiple raster stores defined, the raster analysis service will randomly pick one as the output location.

4. Federate your ArcGIS Server site with your portal.
5. Update your federated Image Server to perform as your portal's image hosting server.
   1. Sign in to the portal website as an administrator and go to Organization > Edit Settings > Servers
   2. From the Image Hosting section of the page, click the drop-down list and choose the ArcGIS Image Server site that you federated with the portal. If you added multiple ArcGIS Image Server sites to your portal, only one can be designated as the image hosting server.
   3. Click Save.
6. Make sure you have increased the SOC maximum heap size to 128 MB for ArcGIS Image Server. See steps 9a through 9f in the Set up an ArcGIS raster analysis server section above.

Note:

If you want to deploy your image hosting server in the cloud with Amazon Web Services, see ArcGIS Enterprise Cloud Builder CLI for AWS. If you want to deploy your raster analysis server in the cloud with Microsoft Azure, see Deploy ArcGIS Enterprise on Microsoft Azure.

Minimum raster analysis system

If an organization has resources and limited conflicts in terms of competing raster analysis processing and image service distribution resources, the raster analysis server and image hosting server functionality can be combined and hosted on one server. In this implementation, the raster analysis and image processing operations are executed on an ArcGIS Image Server site configured as the raster analysis server, while the results are managed and published through the same ArcGIS Image Server site. The ArcGIS Image Server site does the work of processing analytics requests, storing the results, and returning results to members in the Portal for ArcGIS site.

A diagram of the minimum raster analysis system is shown below.

The raster analysis and output image hosting server requires an ArcGIS Image Server license.

The instructions to set up and configure the minimum raster analysis system are detailed below in the Minimum setup for a raster analysis system section.

Minimum setup for a raster analysis system

The following instructions may require changes to the way you've deployed ArcGIS in your organization; review them carefully before proceeding. To configure the portal to perform image hosting services, follow the steps below.

1. Install ArcGIS Server.
2. Authorize ArcGIS Server with an ArcGIS Image Server license.
3. Log in to ArcGIS Server Manager and browse to Site > Data Stores. Register a new raster store.

   The raster store is a location for output from raster analysis and should not be used as an input data directory.

   Note:

   If there are multiple raster stores defined, the raster analysis service will randomly pick one as the output location.

4. Federate your ArcGIS Server site with your portal.
5. Choose your federated Image Server to perform as your portal's raster analysis server.
   1. Sign in to the portal website as an administrator and go to Organization > Edit Settings > Servers.
   2. From the Raster Analysis section of the page, click the drop-down list and choose the server site that you federated with the portal. If you added multiple Image Server sites to your portal, only one can be designated as the raster analysis server.
   3. Click Save.
6. Assign the minimum permissions needed to perform raster analysis.
   1. While still signed in to the portal website as an administrator, go to Organization > Role.
   2. Beside the role that will be performing raster analysis, click Edit.
   3. Check the boxes for the minimum privileges for publishing under Content, which are Create, update, and delete and Publish hosted feature layers.

      

   4. Check the Raster Analysis privilege box under Content and Analysis.

      

7. Make sure you have increased the SOC maximum heap size to 128 MB for ArcGIS Image Server. See steps 9a through 9f in the Set up an ArcGIS raster analysis server section above.