Pooling

All services published with ArcGIS 10.1 for Server and later releases, are pooled. This means that instances of the service can be shared between multiple application sessions.

An application that uses an instance of a pooled service only uses it for the amount of time it takes to complete one request (for example, to draw a map or geocode an address). After the request is completed, the application releases its reference to the service and returns it directly to the available pool of instances.

Recycling

Service recycling allows services that have become unusable to be destroyed and replaced with fresh services; recycling also reclaims resources taken up by stale services.

Services are typically shared between multiple applications and users of those applications. Through reuse, a number of things can happen to a service to make it unavailable for use by applications. For example, an application may incorrectly modify a service's state, or an application may incorrectly hold a reference to a service, making it unavailable to other applications or sessions. In some cases, services may become corrupted and unusable. Recycling allows you to keep the pool of services fresh and cycle out stale or unusable services.

During recycling, the server destroys, then re-creates, each instance in the service configuration. Recycling occurs as a background process on the server. Although you will not see anything on your screen notifying you that recycling is occurring, you can see the events associated with recycling in the log files.

Recycling destroys and re-creates all running instances of a service, whether or not those instances are above the minimum specified. To periodically return the number of running instances to the minimum specified, the service must be stopped and restarted. A good way to automate this process is to create a Python, shell, or Windows batch script that executes a custom ArcGIS Server administrative API command line executable file. This custom executable file would take the server name, service name, service type, and whether the service should be started or stopped as command line arguments.

The time between recycling events is the recycling interval. The default recycling interval is 24 hours, which you can change on the Service Editor dialog box. You can also choose the time that the configuration is initially recycled. From that time forward, recycling will occur each time the recycling interval is reached.

Services are recycled one instance at a time to ensure that instances remain available and to spread out the performance hits caused by creating a new instance of each service. Recycling occurs in random order; however, instances of services in use by clients are not recycled until released. In this way, recycling occurs without interrupting the user of a service.

If there are not enough services available during recycling, a request is queued until an instance becomes available. If the service's maximum wait time is reached during this period, the logs record the same message that they normally would.

Isolation

When you create a service, you specify the minimum and maximum number of instances you want to make available. These instances run on the container machines within processes. The isolation level determines whether these instances run in separate processes or shared processes.

With high isolation, each instance runs in its own process. If something causes the process to fail, it will only affect the single instance running in it.

In contrast, low isolation allows multiple instances of a service configuration to share a single process, thus allowing one process to handle multiple concurrent, independent requests. This is often referred to as multithreading.

With low isolation, a default of 8 instances and a maximum of 24 instances of the same service configuration can share a process. You can set the value for Instances per process on the Processes tab of the Service Editor. When this number of instances of a particular service is created, the server starts an additional process for the next group of instances, and so on. As instances are created and destroyed, they vacate and fill spaces in those running processes.

The advantage of low isolation is that it increases the number of concurrent instances supported by a single process. Using low isolation can use slightly less memory on your server. However, this improvement comes with some risk. If a process experiences a shutdown or crash, all instances sharing the process are destroyed. It is strongly recommended that you use high isolation.

Checking for invalid data connections

When a service instance sits idle, it can be difficult for a server administrator to determine if connections to the source data are being successfully maintained. ArcGIS Server has built-in mechanisms to check for invalid connections to enterprise, workgroup, or desktop geodatabase geodatabases. These checks prevent your service from appearing unresponsive after a connection to enterprise, workgroup, or desktop geodatabase is dropped or interrupted.

You can enable the data connection validity checks by opening the Processes tab of the Service Editor dialog box in either ArcGIS for Desktop or Manager and checking the check box Periodically check and repair data connections for idle instances. You also need to specify an interval in minutes at which service connections will be automatically validated (and repaired if needed). The default of 30minutes is usually appropriate.

Enabling these checks may also help you if firewalls are closing ports to enterprise, workgroup, or desktop geodatabase after your services sit idle for a certain period of time. In this situation, your choice of time interval may be influenced by firewall timeout settings.

Timeouts

Understanding the various available service timeout values can help you keep your services working and available. These values are available on the Pooling tab of the Service Editor dialog box.

Once a client gets a reference to a service, it uses the service for a certain period of time before releasing it. The amount of time between when a client gets a reference to a service and when it releases it is the usage time. To ensure that clients don't hold references to services for too long (that is, they don't correctly release services), each service has a maximum time a client can use a service. If a client holds on to a service for longer than the maximum usage time, the service is automatically released and the client loses its reference to the service.

Maximum usage time also protects services from being used to do larger volumes of work than the administrator intended. For example, a service that is used by an application to perform geodatabase checkouts might have a maximum usage time of 10 minutes. In contrast, a service with one layer that is only used to draw maps in an application might have a maximum usage time of 1 minute.

When the maximum number of instances of a service is in use, a client requesting a service is queued until another client releases one of the services. The amount of time it takes between a client requesting a service and getting a service is the wait time. Each service has a maximum time a client will wait to get a service. If a client's wait time exceeds the maximum wait time for a service, the request times out.

A third timeout dictates the maximum time an idle instance can be kept running. When services go out of use, they are kept running on the server until another client needs the instance. A running instance that is not in use still consumes some memory on the server. You can minimize your number of running services and therefore conserve memory by shortening this idle timeout, the default of which is 1,800 seconds (30 minutes). The disadvantage of a short idle timeout is that when all running services time out, subsequent clients need to wait for new instances to be created.

When service instances are created in the GIS server, either as a result of the server starting or in response to a request for a server by a client, the time it takes to initialize the service instance is referred to as its creation time. The GIS server maintains a startup timeout that dictates the amount of time that can elapse on a startup attempt before the GIS server assumes its startup is hanging and cancels the creation of the service instance. The default is 300 seconds (5 minutes).

The GIS server maintains statistics both in memory and in its logs about wait time, usage time, and other events that occur within the server. The server administrator can use these statistics to determine if, for example, the wait time for a service is high, which may indicate a need to increase the maximum number of instances for that service.

There may be additional timeouts encountered in your architecture that will cause discrepancies between the service timeout values you specify and the actual timeouts experienced by clients. For example, the web server hosting ArcGIS Web Adaptor or a network load balancer may impose timeouts that affect your services.

Limiting what users can do with a service

To make it easy to control how your web services are used, each type of service has a set of allowed operations. Each operation consists of a set of methods that can be enabled or disabled as a group. Clients of the web service can only call the methods of the operations that have been allowed.

Suppose you wanted to allow consumers of a mapping web service to draw the map but not to query the data sources of the map's layers. You would then need to disable the Data operation and ensure that the Map operation is allowed.

Feature services are of special interest in this discussion because they are used for performing web-based editing of GIS data. Feature services have an additional set of operations that can be used to restrict editing functionality. You can enable or disable these in the Feature Access tab of the Service Editor dialog box in ArcGIS for Desktop. You can also prevent users from editing features they did not originally create by enforcing ownership-based access control.

See the Types of services section of the help to learn about the allowed operations on various service types.