ArcGIS Enterprise is a collection of software components that can be deployed in many configurations based on an organization's requirements and desired workflows. At each varied capacity and configuration, the software requires stability from the underlying operating systems and infrastructure to operate at its highest potential. The following recommendations and techniques can be used to plan for and deliver stable, optimized, and highly functional production sites at every scale.
Planning for deployment
Prior to deploying ArcGIS Enterprise, consider these recommendations.
Plan for and allocate storage
After a period of routine use, ArcGIS Enterprise will require additional storage space over time to support workflows such as publishing services and sharing content. The amount of storage space needed will vary and depend on several factors, such as the following:
- Types of published services
- Amount and types of content and items uploaded to the organization
- Data storage to support published services (when data is copied to the server during publishing as opposed to referenced directly in a shared location)
While disk space increases over time to support these workflows, the size of scheduled backups will scale proportionally. To plan for sufficient backup storage, it is recommended that retention policies are used for stored backups. For example, in the event when a collection of incremental backups is only useful when coupled with the last full backup in the chain, they should be removed when that chain falls outside of retention requirements. Retention policies should align with organizational requirements to prevent data loss and other applicable regulatory requirements.
Plan for maintenance windows
Prior to deploying ArcGIS Enterprise, it is recommended that administrators determine proper outage requirements and business dependencies for the ArcGIS Enterprise organization. Considerations should include operating system (OS) and software level patching, as well as configuration changes that require downtime to the organization. Planning for regularly scheduled maintenance windows will allow administrators to set availability expectations across the organization and reduce the frequency of unexpected downtime for emergency change requests. Administrators can customize a notification banner to alert the organization's members with details on when these maintenance windows occur.
Consider implementing tiered environments
Proper implementation of, and ongoing comparison between, multiple tiered environments (for example, development, staging, and production) has proven to be an extremely valuable tool in terms of reduction of production-down incidents. Many times, the different tiers will become out of sync in terms of system-level configuration, so ensuring that proper change management is carried out will increase the overall stability of the production tier, while allowing for proper vetting of configuration changes in the lower environments. Using these tiered environments to not only perform acceptance and load testing of content is important, but installation of OS and ArcGIS Enterprise patches should be installed following the tiered approach as well to minimize potential for disruption of the production environment.
Once the deployment is in use across the organization, consider the following recommendations.
Each ArcGIS Enterprise component generates log files that can be used to identify and troubleshoot issues:
It is recommended that administrators review generated logs for all entries logged at SEVERE or WARNING level, as they may indicate software functionality issues that must be addressed.
Apply updates and patches
Regular patching is an important part of maintaining secure, up-to-date, stable environments. In terms of scheduling patches, proper backup procedures will help the organization to recover in situations in which the patch introduces adverse behavior, or some sort of failure is experienced. Patching also requires some sort of validation to take place to confirm normal operations of services on the patched machines following the installation. Patching of systems are split into two broad categories: operating system patches and software patches. The release of operating system patches is typically monthly for Windows or more frequent for individual packages on Linux, while the release of ArcGIS Enterprise patches depends on the version's maturity. Administrators should regularly check for updates using the included patchnotification utility and plan to install during regular maintenance windows.
Installation of patches can vary in time, but typically range from 1–2 hours for Portal for ArcGIS and 15–30 minutes for ArcGIS Server and ArcGIS Data Store. Checking the CPU utilization for the running patch process (typically msiexec.exe on Windows or a bash process running the applyPatch script on Linux) can be a good method to assess progress of the running patch process. Organizations that have established operating system patching cycles can use those schedules to shape when maintenance windows need to be scheduled and when is a suitable time to apply ArcGIS Enterprise patches as well.
When patching highly available deployments, it is recommended that no two (or more) machines within a site should be patched simultaneously. When patching environments where ArcGIS Enterprise components are installed on separate machines, the general rule is to patch machines in this order:
- ArcGIS Data Store
- ArcGIS Server
- Portal for ArcGIS
This recommended order is due to dependencies between published services and the availability of back-end data stores.
When an organization's TLS/SSL certificate expires, members may lose connection and the organization may become inaccessible. To alleviate this, administrators can maintain a schedule to request and apply replacement certificates to avoid disruption or loss of connectivity. Typically, certificates expire annually or every two years if issued by a public certifying authority, while internal certificates can extend much longer but will eventually need to be rotated with a renewed certificate. Proper planning allows not only for the renewal to occur before the expiration date, but also updated within the web servers during a maintenance window to avoid interruption of service.
When employing SAML (Security Assertion Markup Language) authentication, consideration should be made for the signing and encryption of certificates that are used to validate the SAML request and response with the providers. Maintaining a record of those certificate renewal/rotation requirements will prevent disruption to the organization.
The authentication method your organization uses may impose additional considerations for administering your ArcGIS Enterprise deployment. When connected to Windows Active Directory or LDAP (Lightweight Directory Access Protocol), a user password is in place to establish a connection to the identity store. If this password expires, the identity store connection can disconnect and prevent users from authenticating. This account should be managed as a proactive rotation of the user/password during a maintenance window.
In Windows environments, the user account that runs the Windows service for each ArcGIS Enterprise component should be managed to prevent passwords from expiring. Use of a group managed service account (gMSA) avoids the need to rotate the service account password since it is handled by Windows Active Directory.
Administrators should also be aware of when database user passwords may be expiring, as this can directly affect the availability of the ArcGIS Server services. Database user passwords can be updated by importing a new database connection file to the existing registered database connection in either ArcGIS Server Manager, ArcGIS Desktop, or ArcGIS Pro. Services that depend on those database connections should be stopped prior to updating the database user's password to avoid the potential for locked accounts due to authentication requests containing the expired password.
To ensure sufficient and adequate machine resources are available to your ArcGIS Enterprise deployment, it is important that you perform ongoing monitoring of CPU, RAM, and disk usage. By identifying typical usage, you can observe trends and detect anomalies and fine-tune the resources accordingly on each participating machine over time. Administrators can configure alerts to notify IT and GIS administrators when certain requirements are at risk, such as when disk space is below a certain threshold for free space or when CPU utilization has spiked beyond an expected duration. These alerts provide an added safety measure for administrators to proactively implement corrective actions and to prevent outages due to increases in demand.
Manage data sources
ArcGIS Server service performance is dependent on the underlying database instances that host the referenced data in published services. To optimize map and feature services for query efficiency, confirm there are no bottlenecks on the data tier that would negatively impact response times. The same rules outlined above for machine resources apply to the back-end DBMS instances and can be supplemented with tools such as database tracing. Some database management system instances have a maximum number of connections allowed, which could become important during scaling events of your ArcGIS Server services or the addition of a machine to the site. Some organizations assign this level of monitoring to their database administration team while in other organizations the responsibility may fall to the IT team that oversees the infrastructure.
Check antivirus and antimalware exclusions
While exclusions defined for a security product are typically static, it is useful to check with the team responsible for those products on a semi-regular basis, annually for example, to confirm that the exclusions are still in place and do not negatively impact performance of ArcGIS Enterprise software. Having the checkup conversation can also translate into an understanding of future updates or software replacements that may affect the operation of the existing deployments.
Audit file system permissions
During installation, each ArcGIS Enterprise component is configured with permissions to run with a specified service account. If these permissions are modified, outages or software malfunctions may occur. Updates to these permissions should be rare and controlled for a least-privilege access model as well as for when patches are applied within tiered environments. If file permissions are observed to change frequently, an audit of those changes should be enabled to prevent interruptions in the software.