Configuring a highly available portal

In this topic

Prerequisites for configuring a highly available portal
Configuring a highly available portal
Check the status of a highly available portal
Delete a highly available portal
Upgrade a highly available portal
Troubleshooting
Error messages

In an organization where downtime must be minimized, Portal for ArcGIS needs to be configured in a highly available fashion. The way you achieve this is by installing the software on two machines and configuring them using an installed utility.

Configuring high availability is an advanced task that requires an extensive understanding of portal administration, scripting, and networking. Before you install and configure Portal for ArcGIS, you'll be required to configure your organization's load balancer to forward requests to the portal software. Additionally, you'll also need to set up a file server to contain the portal's content directory. It's recommended that you coordinate with your organization's information technology staff so they understand the requirements for configuring a highly available portal.

High availability deployment — In this architecture, each portal is installed on its own machine and references a shared content directory. Administrators connect to the site through the network load balancer. The content directory is shared to both portal machines through a file server.

In this architecture, a network load balancer or reverse proxy server is configured with both portal machines and acts as a gateway to the organization. If you intend to use web-tier authentication, ArcGIS Web Adaptor is required. The Web Adaptor can be installed in front of the load balancer or on both portal machines. If you're not using web-tier authentication, ArcGIS Web Adaptor is not required.

Both portal machines include databases that store information about content. The database on the first machine replicates changes to the database on the second machine. An index service keeps users and item searches in sync between both machines.

If you'll be using ArcGIS Server with your highly available portal, it's recommended that you use the network load balancer to balance requests between the two components. This ensures that requests from ArcGIS Server are sent to the portal in a highly available fashion. If you don't want to use your network load balancer for internal communication with ArcGIS Server, you can add a second network load balancer that is only available internally. Alternatively, you can place two or more Web Adaptors in front of the network load balancer. In this fashion, all public traffic is handled by the Web Adaptors, and all internal traffic is handled by the load balancer.

Prerequisites for configuring a highly available portal

To configure high availability for your portal, the following components are required:

Network load balancer (NLB)—A third-party component that uses a distribution algorithm to load balance network traffic across both portal machines, helping to enhance the scalability and availability of the portal. It must provide high availability by detecting machine failures and automatically redistributing traffic to the available portal machine. The NLB context name must be set to arcgis (for example, https://nlb.domain.com/arcgis).
Highly available file server—A third-party component that stores and shares the portal's content directory. The file directory you select must be accessible by both machines and the account that will be used to install Portal for ArcGIS. The same account must be used on both portal machines.
Two Portal for ArcGIS machines—You'll need two separate machines installed with Portal for ArcGIS to configure high availability. These machines must meet the minimum operating system requirements, and the same account must be used to install the portal software.

The following components are optional:

ArcGIS Web Adaptor—An optional component included with Portal for ArcGIS that you can use to provide web-tier authentication. The Web Adaptor can be installed in front of the load balancer or on both portal machines. If you're not using web-tier authentication, ArcGIS Web Adaptor is not required. To learn more, see About ArcGIS Web Adaptor.
ArcGIS Server—An optional component that makes GIS web services available to others in your organization. Using ArcGIS Server with your portal provides many benefits as described in About using your server with Portal for ArcGIS. If you'll be federating your ArcGIS Server site with your portal, there are some additional steps you'll need to perform as described below.

Configuring a highly available portal

To configure your portal to be highly available, follow these steps.

Note:

To upgrade your highly available portal to 10.3.1, follow the steps in the upgrade section below.

Tip:

You can obtain the status of the high availability configuration and delete the high availability configuration at any time using a command line utility. For information on issues you may encounter when configuring a high availability configuration, see Troubleshooting below.

Step 1: Configure a network load balancer

Set the NLB context name to arcgis (for example, https://nlb.domain.com/arcgis). If you use a context other than arcgis, failures will occur in the highly available portal.
Configure HTTPS on the network load balancer. This is necessary as Portal for ArcGIS requires HTTPS for some communication. Consult the product documentation for your NLB to learn how to set up HTTPS.
Configure your NLB to load balance requests to both of your portal machines (p1.domain.com and p2.domain.com).
- If you're not using web-tier authentication, configure the NLB to load balance requests to ports 7080 (HTTP) and 7443 (HTTPS). By default, Portal for ArcGIS uses these ports for communication; you'll need to include these ports as part of the configuration. For example, on Apache, the ports are specified in the httpd.conf and httpd-ssl.conf configuration files. To learn more, see Ports used by Portal for ArcGIS.
- If you're using web-tier authentication (and installing ArcGIS Web Adaptor on each portal machine), configure the NLB to load balance requests to ports 80 (HTTP) and 443 (HTTPS). You can only use ArcGIS Web Adaptor with web server ports 80 and 443. Using different ports is not supported.
In the NLB configuration, set an X-Forwarded-Host header. Portal for ArcGIS expects to see this property set in the header sent by the NLB and will return requests to the NLB that match the NLB's URL. For example, a request to the ArcGIS Portal Directory (https://nlb.domain.com/arcgis/sharing/rest) will be returned to the client as the same URL. If the property is not set, Portal for ArcGIS may return the URL of the internal machine where the request was directed (for example, https://p1.domain.com/arcgis/sharing/rest instead of https://nlb.domain.com/arcgis/sharing/rest). This is problematic, as clients will not be able to access this URL (commonly noted as a browser 404 error). Also, the client will have some knowledge about the internal machine.

Step 2: Set up the portal content directory on a file server

In a highly available configuration, the portal's content directory is shared between both machines. You must set up the content directory so it is accessible by both machines and the account that will be used to install Portal for ArcGIS. The same account must be used on both portal machines.

On the file server, create a directory for the portal's content directory and share it so that it can be accessed by both portal machines. For example, /net/share/portal/content.
Grant the account that will be used to install Portal for ArcGIS 700 permissions to the directory.
Verify that the directory can be accessed by the account on both machines.

Step 3: Install and configure the first portal machine

On the first portal machine, open the ports described in Ports used by Portal for ArcGIS. Additionally, open ports 57800, 57900, 57950, and 57975. These ports are used by an index service to keep users and item searches in sync between both portal machines.
Install Portal for ArcGIS on the first machine. For full instructions, see Installing Portal for ArcGIS.
Open the portal website and create the initial administrator account. The URL to the website is formatted https://p1.domain.com:7443/arcgis/home. The initial administrator is not an operating system account, and it has no relation to the account used to install Portal for ArcGIS.
When the account is created, you'll see a message stating that the portal will be restarted. Click OK.
Copy all the contents under the local content directory to the directory you specified on your file server. For example, open <Portal for ArcGIS installation directory>/arcgis/portal/usr/arcgisportal/content and copy its contents to /net/share/portal/content.

Step 4: Install and configure ArcGIS Web Adaptor

If you'll be using web-tier authentication, you're required to install and configure ArcGIS Web Adaptor. The Web Adaptor can be installed in front of the load balancer or on both portal machines. You can only use the Web Adaptor with web server ports 80 and 443. Using different ports is not supported. If you're not using web-tier authentication, ArcGIS Web Adaptor is not required.

If you're installing the Web Adaptor in front of your NLB

If you're installing the Web Adaptor in front of your NLB, it's recommended that you place the Web Adaptor on a highly available web server. This software component is a single-point of failure for your portal deployment.

Install ArcGIS Web Adaptor on a web server machine located in front of your network load balancer. For full instructions, see the installation topic for IIS, Java (Windows) or Java (Linux).
Configure the Web Adaptor. When specifying the Portal URL, enter the URL of the network load balancer, for example, https://nlb.domain.com/arcgis. For instructions, see the configuring topic for IIS, Java (Windows), or Java (Linux).

If you're installing the Web Adaptor behind your NLB

If you're installing the Web Adaptor behind your NLB, you'll need to place the Web Adaptor on both portal machines. This method provides a highly available deployment of the Web Adaptor.

On the first portal machine, install ArcGIS Web Adaptor. For full instructions, see the installation topic for IIS, Java (Windows) or Java (Linux).
Configure the Web Adaptor with the first portal machine. When specifying the Portal URL, enter the URL of the first portal machine, for example, https://p1.domain.com:7443/arcgis. For instructions, see the configuring topic for IIS, Java (Windows), or Java (Linux).
On the first portal machine, open the ArcGIS Portal Directory (https://p1.domain.com:7443/arcgis/portaladmin) and log in using the initial administrator account.
Click System > Web Adaptors > Configuration and copy and save the Shared key (for example, LLjb9UMC3tdqdAZGa+107ckUe9dfeONJJLjQ/CA9ERA=).
Replace the existing shared key on the second portal machine with the key you saved from the first portal machine. Full instructions are provided as part of Step 6.
Install and configure ArcGIS Web Adaptor on the second portal machine. Full instructions are provided as part of Step 6.

Step 5: Run the high availability utility on the first portal machine

A command line utility is provided that does the work of configuring your portal machine for high availability. You'll need to provide some information about your deployment to execute the utility.

On the first portal machine, log in as the account used to install the portal software and open <Portal for ArcGIS installation directory>/tools/portalha.
From the command line, run the portalha.sh tool with the -c command (for example, ./portalha.sh -c).
Provide the following information in the command line:
- Shared content folder—The content directory you set up on your file server, for example, /net/share/portal/content.
- Public portal URL—The public URL that clients use to access your organization. Depending on how you configured your highly available portal, this could be the NLB, ArcGIS Web Adaptor, or another machine that serves as the gateway to your organization. For example, if you installed ArcGIS Web Adaptor in front of your NLB, the public portal URL is the URL of the machine hosting the Web Adaptor (https://webadaptor.domain.com/arcgis).
- Private portal URL—The private URL that is used to locally access your organization. Commonly, this is the NLB installed in front of your portal machines (https://nlb.domain.com/arcgis). If you've configured a second NLB to handle internal communication between your highly available portal and ArcGIS Server, specify the URL to the internal NLB (https://internalnlb.domain.com/arcgis).
- Other portal machine name—The name of the other portal machine in the high availability deployment, for example, p2.domain.com.
Type Y and press Enter to run the utility.

Step 6: Install and configure the second portal machine

On the second portal machine, open the ports described in Ports used by Portal for ArcGIS. Additionally, open ports 57800, 57900, 57950, and 57975. These ports are used by an index service to keep users and item searches in sync between both portal machines.
Install Portal for ArcGIS on the second machine. When installing, use the same account that installed the software on the first machine. Also, specify the exact same installation and content directory locations you selected for the first machine. For full instructions, see Installing Portal for ArcGIS.
Open the portal website and create the initial administrator account. Specify the exact same user information used to create the account on the first machine. The URL to the website is formatted https://p2.domain.com:7443/arcgis/home. The initial administrator is not an operating system account, and it has no relation to the account used to install Portal for ArcGIS.
When the account is created, you'll see a message stating the portal will be restarted. Click OK.
If you're using web-tier authentication and you're installing the Web Adaptor behind your NLB, install and configure ArcGIS Web Adaptor on the second portal machine. You can only use the Web Adaptor with web server ports 80 and 443. Using different ports is not supported. If you've installed the Web Adaptor in front of your NLB, skip this step.
1. On the second portal machine, install ArcGIS Web Adaptor. When installing, give it the same name as the first ArcGIS Web Adaptor. For example, if your first Web Adaptor is named arcgis, name the second Web Adaptor arcgis. For full instructions, see the installation topic for IIS, Java (Windows), or Java (Linux).
2. On the second portal machine, open the ArcGIS Portal Directory (https://p2.domain.com:7443/arcgis/portaladmin) and log in using the initial administrator account.
3. Click System > Web Adaptors > Configuration > Update Configuration.
4. In the JSON configuration text box, replace the existing shared key with the key you saved from the first portal machine (as part of Step 4), for example:
  {"sharedKey": "LLjb9UMC3tdqdAZGa+107ckUe9dfeONJJLjQ/CA9ERA="}
5. Click Update Web Adaptors Configuration.
6. Configure the Web Adaptor with the second portal machine. When specifying the Portal URL, enter the URL of the second portal machine, for example, https://p2.domain.com:7443/arcgis. For instructions, see the configuring topic for IIS, Java (Windows), or Java (Linux).

Step 7: Run the high availability utility on the second portal machine

A command line utility is provided that does the work of configuring your portal machine for high availability. You'll need to provide the location of the content directory on your file server when running the utility on the second machine.

On the second portal machine, log in as the account used to install the portal software and open <Portal for ArcGIS installation directory>/tools/portalha.
From the command line, run the portalha.sh tool with the -j command (for example, ./portalha.sh -j).
Provide the following information in the command line:
- Shared content folder—The content directory you set up on your file server, for example, /net/share/portal/content.
Type Y and press Enter to run the utility.

Your portal is now configured for high availability. You can now optionally federate an ArcGIS Server site with your portal to provide additional functionality and sharing capabilities as described in About using your server with Portal for ArcGIS. To get started, proceed with the steps in the following section.

Step 8: Federate an ArcGIS Server site with your portal (optional)

Follow the instructions in Federating an ArcGIS Server site with your portal to federate the server with your highly available portal deployment.
When federation is complete, open a web browser and log in to the ArcGIS Server Administrator Directory as an Administrator and click security > config > update. The URL is typically available at https://webadaptor.domain.com/arcgis/admin.
In the input box, update the privatePortalURL property in the portal properties section to match the Private portal URL you specified in Step 5.
Restart all of the GIS server machines in your ArcGIS Server site.

Check the status of a highly available portal

To check the status of a high availability configuration, use the command line utility and follow these steps:

On any machine in the high availability configuration, log in as the account used to install the portal software and open <Portal for ArcGIS installation directory>/tools/portalha.
From the command line, run the portalha.sh tool with the -s command (for example, ./portalha.sh -s).
Review the messages in the tool to obtain the current status of the high availability configuration.

Delete a highly available portal

To delete the high availability configuration, use the command line utility and follow these steps:

On the first portal machine, log in as the account used to install the portal software and open <Portal for ArcGIS installation directory>/tools/portalha.
From the command line, run the portalha.sh tool with the -d command (for example, ./portalha.sh -d).
Type Y and press Enter to delete the configuration.

Upgrade a highly available portal

Upgrading a highly available portal to 10.3.1 involves the following steps described in the sections below. The steps vary between the first and second machines in the configuration. You must follow the steps exactly as described below to upgrade your highly available configuration to 10.3.1.

Upgrade the first portal machine

Delete the high availability configuration as described in Delete a highly available portal above.
Back up the portal content directory you set up in Step 2.
Place a copy of the portal content directory in a local directory on first portal machine (for example, /home/admin/portal/content).
Grant the account used to install Portal for ArcGIS 700 permissions to the local directory.
Configure the portal to use the local content directory:
1. Sign in to the ArcGIS Portal Directory as an Administrator of your organization. The URL is in the format https://webadaptor.domain.com/arcgis/portaladmin.
2. Edit the directory location by clicking System > Directories > content > Edit Directory.
3. Specify the location of the new content directory in the Path text box (for example, /home/admin/portal/content).
4. Click Edit Directory to apply your changes.
5. Click System > Directories > content and verify that the new directory is being used.
Delete the portal content in the shared directory.
Install Portal for ArcGIS 10.3.1. You do not need to uninstall the software first; run the 10.3.1 setup on the machine to upgrade the portal. For full instructions, see Installing Portal for ArcGIS.
Open the portal website and create the initial administrator account. The URL to the website is formatted https://p1.domain.com:7443/arcgis/home. This triggers the upgrade. Do not interrupt this process. The initial administrator is not an operating system account, and it has no relation to the account used to install Portal for ArcGIS.
When the upgrade completes, you'll see a message stating that the portal will be restarted. Click OK.
Copy all of the contents under the local content directory (from Step 3) to the directory you specified on your file server. For example, open /home/admin/portal/content and copy all of its contents to /net/share/portal/content.
Run the high availability utility on the first portal machine (as described in Step 5 above).
On the first portal machine, open the ArcGIS Portal Directory and sign in with the initial administrator account. The URL is formatted https://p1.domain.com:7443/arcgis/portaladmin.
Click System > Indexer > Reindex.
Click the Mode drop-down list and select Full.
Click Reindex. This step will complete the upgrade of your portal. Depending on the number of users and volume of content in your portal, it will take some time for the reindex to complete. Do not interrupt the reindex process. You can monitor the indexing status by opening a new browser window (or tab), browsing to System > Indexer > Index Status, and refreshing the page. When the store and index counts are equal, the reindex and upgrade is complete.

Uninstall and reinstall the portal on the second machine

After completing the upgrade steps on the first portal machine, follow the steps below to upgrade the second portal machine.

Uninstall Portal for ArcGIS. See Uninstalling Portal for ArcGIS for full instructions.
Install Portal for ArcGIS 10.3.1. For full instructions, see Installing Portal for ArcGIS.
1. After the installation completes and you've finished the software authorization, stop Portal for ArcGIS. To do this, stop the Portal for ArcGIS service under Control Panel > Administrative Tools > Services.
2. Grant the account you specified when setting up the portal content directory Full control permissions to the following directories:
  - Portal for ArcGIS installation directory, for example, C:\Program Files\ArcGIS\Portal
  - ArcGIS Portal folder, for example, C:\arcgisportal
3. In the Services panel, right-click the Portal for ArcGIS service and choose Properties.
4. Click the Log On tab and choose This account from the log on as options.
5. Specify the account name and credentials that you specified when setting up the portal content directory on your file server. Optionally, you can click Browse to specify the account from the Select User dialog box. Once you've specified the account name and password, click Apply.
6. Click the General tab and click Start. The Portal for ArcGIS service is started and is now running as the account you specified.
7. Click OK.
Open the portal website and create the initial administrator account. Specify the exact same user information used to create the account on the first machine. The URL to the website is formatted https://p2.domain.com:7443/arcgis/home. The initial administrator is not an operating system account, and it has no relation to the account used to install Portal for ArcGIS.
When the account is created, you'll see a message stating that the portal will be restarted. Click OK.
Run the high availability utility on the second portal machine (as described in Step 7 above).
On the second portal machine, open the ArcGIS Portal Directory and sign in with the initial administrator account. The URL is formatted https://p2.domain.com:7443/arcgis/portaladmin.
Click System > Indexer > Reindex.
Click the Mode drop-down list and select Full.
Click Reindex. This step will complete the upgrade of your portal. Depending on the number of users and volume of content in your portal, it will take some time for the reindex to complete. Do not interrupt the reindex process. You can monitor the indexing status by opening a new browser window (or tab), browsing to System > Indexer > Index Status, and refreshing the page. When the store and index counts are equal, the reindex and upgrade is complete.

Upgrade remaining ArcGIS components

Upgrade the remaining ArcGIS components in your deployment to 10.3.1. These must be updated to 10.3.1 in order to be used with your high availability configuration, for example

ArcGIS Web Adaptor (you must uninstall and reinstall to upgrade)
ArcGIS Server (run the 10.3.1 setup to upgrade)
ArcGIS Data Store (run the 10.3.1 setup to upgrade)

Tip:

Upgrade Portal for ArcGIS outlines additional considerations when upgrading your deployment to 10.3.1.

Troubleshooting

Common problems and solutions

I updated the security configuration of my highly available portal and one of the portal machines cannot connect to my user or group store. What's going on?

When configuring security for your portal, you're required to restart each portal machine in order to apply the configuration updates correctly. Restart each portal machine to work around this issue. See Stopping and starting the portal for full instructions.

Error messages

The following table lists common errors you may encounter when attempting to configure a high availability configuration using the command line utility. While most messages are self-explanatory, some include a description to provide further information and guidance for troubleshooting.

Message variables, such as a machine name, directory, URL, and so on, appear in angle brackets (<variable>) in the text below.

Message Text	Description
The machine <machine> is already a peer machine. Specify a different peer machine.	The machine you specified to create or join the high availability configuration is already part of the configuration. Verify the machine you specified is the correct machine and try again.
High availability architecture has already been created. To obtain the current status of the high availability configuration, run portalha -s.	The high availability architecture has already been created on the machine you specified. Verify the machine you specified is the correct machine and try again.
The shared content directory cannot be empty. It must contain a copy of the content from the first portal machine.	After you install and configure Portal for ArcGIS on the first machine, you're required to copy all of the contents under the local content directory to the directory you specified on your file server. For example, the content from <Portal for ArcGIS installation directory>/arcgis/portal/usr/arcgisportal/content must be copied to /net/share/portal/content. Verify the shared content directory contains a copy of the content from the first portal machine.
Cannot access the shared content directory <directory>. Verify the file path is correct and the account used to install Portal for ArcGIS has 700 permissions to the directory.	In a highly available configuration, the portal's content directory is shared between both machines. You must set up the content directory so it is accessible by both machines and the account used to install Portal for ArcGIS. The account must have 700 permissions to the directory.
The account used to install Portal for ArcGIS does not have 700 permissions to the shared content directory <directory>. Grant the account 700 permissions to the directory.
Cannot find the items folder under the shared content directory <directory>. Verify the directory contains a copy of the items folder from the first portal machine.	After you install and configure Portal for ArcGIS on the first machine, you're required to copy all of the contents under the local content directory to the directory you specified on your file server. For example, the content from <Portal for ArcGIS installation directory>/arcgis/portal/usr/arcgisportal/content must be copied to /net/share/portal/content. Verify that the shared content directory contains a copy of the items folder from the first portal machine.
The public portal URL <URL> is invalid. Verify the URL you specified is correct.	The public URL that clients use to access your organization. Depending on how you configured your highly available portal, this could be the NLB, ArcGIS Web Adaptor, or another machine that serves as the gateway to your organization. For example, if you installed ArcGIS Web Adaptor in front of your NLB, the public portal URL is the URL of the machine hosting the Web Adaptor (https://webadaptor.domain.com/arcgis).
The public portal URL <URL> is not reachable. Verify the public portal URL is online and accessible.
The private portal URL <URL> is invalid. Verify the URL you specified is correct.	The private URL used to locally access your organization. Commonly, this is the NLB installed in front of your portal machines (https://nlb.domain.com/arcgis). If you've configured a second NLB to handle internal communication between your highly available portal and ArcGIS Server, specify the URL to the internal NLB (https://internalnlb.domain.com/arcgis).
The private portal URL <URL> is not reachable. Verify the private portal URL is online and accessible.
The machine <machine> is not listed in the high availability configuration. Verify that you have specified the correct machine.
The machine <machine> hosting the primary database is not reachable. Verify machine <machine> is running and accessible.
The machine <machine> was not added to the high availability architecture.
The machine <machine> is already configured with the high availability architecture. To obtain the current status of the high availability configuration, run portalha -s.	The high availability architecture has already been created on the machine you specified. Verify that the machine you specified is the correct machine and try again.
The database administrator credentials on machine <machine> are invalid. Verify the credentials specified for this machine match the first portal machine.
The syntax specified for the peer machines <machine> is invalid. Each peer machine must be separated with a comma.	The peer machine list provided includes syntax errors. Verify that each peer machine in the list is separated with a comma.
No peer machines have been specified.
Unrecognized command specified. For help, run portalha -h.
The Portal for ArcGIS installation directory is empty. Verify that Portal for ArcGIS has been installed correctly on the machine.	When you install and configure Portal for ArcGIS on the first machine, a local folder is created called the Portal for ArcGIS installation directory. For example, /home/admin/arcgis/portal. This directory is populated with files and subfolders during the installation. If the installation is incomplete or an error occurred, the directory may be empty. Verify that Portal for ArcGIS has been installed correctly on the machine.


The arcgisportal directory is empty. Verify that Portal for ArcGIS has been installed correctly on the machine.	When you install and configure Portal for ArcGIS on the first machine, a local folder is created called the arcgisportal directory. For example, <Portal for ArcGIS installation directory>/arcgis/portal/usr/arcgisportal/content. This directory is populated with files and subfolders during the installation. If the installation is incomplete or an error occurred, the directory may be empty. Verify that Portal for ArcGIS has been installed correctly on the machine.


The machine <machine> is not configured for high availability architecture.	The machine you specified to delete or check the status of the high availability configuration is not part of the configuration. Verify that the machine you specified is the correct machine and try again.
The high availability architecture has not been created. You must create the high availability architecture before you can delete it.
Portal for ArcGIS on machine <machine> is not running properly. Restart Portal for ArcGIS and try again.
Unable to find the required property <property> in file <file>.	The high availability properties file is damaged or inaccessible. Verify the status of your configuration by running the portalha -s command.
Unable to find the file <file>.	The high availability properties file is inaccessible or was deleted. Verify the status of your configuration by running the portalha -s command.

Feedback on this topic?