Troubleshoot ArcGIS Enterprise on Microsoft Azure

If you encounter issues using ArcGIS Enterprise Cloud Builder for Microsoft Azure, see the log files created in the %temp% directory where Cloud Builder is installed (by default, the directory is C:\Users\<your login>\AppData\local\temp). Additional troubleshooting tips are provided here.

Also note that virtual machines created during a failed deployment process are not automatically deleted. This allows you to connect to the virtual machine and check the Windows event logs or the Cloud Builder deployment log (C:\arcgis\Deploy\deploy-log.txt) to troubleshoot the deployment failure. However, this means you need to clean up these virtual machines yourself after you finish troubleshooting. Use the Microsoft Azure Management Portal to delete machines that have been provisioned before you redeploy a site.

You can also configure Microsoft Azure Monitor Logs to collect ArcGIS Server logs and logs for the Azure virtual machines.

To allow Azure Monitor Logs to gather ArcGIS Server logs (services.log and server.log files), you must define a custom logs data source on the Log Analytics page in the Azure portal. The ArcGIS Server log files are stored in the C:\ArcGIS\serverlogs directory on the ArcGIS Server machines.

ArcGIS Enterprise Cloud Builder for Microsoft Azure

• When creating or upgrading a deployment using Cloud Builder, I receive a time-out message similar to the following: The resource operation completed with terminal provisioning state 'Failed' (Code: ResourceDeploymentFailure)— Provisioning of VM extension '<resrouce name>' has timed out. Extension installation may be taking too long, or extension status could not be obtained. (Code: VMExtensionProvisioningTimeout).
• I receive the message ResourcePurchaseValidationFailed: User failed validation to purchase resources. Error message: 'Legal terms have not been accepted for this item on this subscription. To accept legal terms, please go to the Azure portal and configure programmatic deployment for the Marketplace item or create it there for the first time when I attempt to deploy an ArcGIS Server site or ArcGIS Enterprise site using Cloud Builder.
• I receive an unhandled exception error in Cloud Builder when the Microsoft Azure session token expires.
• I am unable to sign out of Cloud Builder.
• I receive a 401 error when I click a button or link in Cloud Builder.
• I receive a 403 error when I try to create or add to a deployment in Azure using Cloud Builder.
• I receive the error Unable to download deployment content when using Microsoft Az.Storage 6.0.0 Microsoft PowerShell module with artifacts generated by Cloud Builder 11.2 and earlier to create or upgrade a deployment in Microsoft Azure.

Site administration

• I cannot make a remote desktop connection to the Microsoft Azure virtual machine.
• Administrative access is enabled through the web proxy by default, but I don't want it enabled.
• After installing an SSL (TLS) certificate, the URL property on existing web service items—including the utility services that are included with Portal for ArcGIS—is not updated. They still use the original *.cloudapp.azure.com URL.
• I can't open the Microsoft Azure portal, Azure Marketplace, or Azure pricing calculator.
• I receive an invalid token error when I use the webgisdr tool to restore a version 2 (V2) deployment.

Upgrades

• I receive the following error after I upgrade ArcGIS Server: The connection property set was missing a required property or the property value was unrecognized. Connection was attempted with an older version of SQL Server client communications software that is not supported.

Publishing

• Publishing to my portal on Microsoft Azure fails, and I receive the message Publish exception 'Exception: no protocol: null/admin/services/exists'.

General

• I and other portal members intermittently receive HTTP 502 errors when connecting to ArcGIS Enterprise portals on Microsoft Azure.
• Can I access data stored in Microsoft Azure from an ArcGIS client running on premises?

ArcGIS Enterprise Cloud Builder for Microsoft Azure

When creating or upgrading a deployment using Cloud Builder, I receive a time-out message similar to the following: The resource operation completed with terminal provisioning state 'Failed' (Code: ResourceDeploymentFailure)— Provisioning of VM extension '<resrouce name>' has timed out. Extension installation may be taking too long, or extension status could not be obtained. (Code: VMExtensionProvisioningTimeout).

Due to a current restriction in the Microsoft Azure Resource Manager (ARM), templates that do not complete running in under 90 minutes will time out. As a result, deployments that take longer than 90 minutes to create or upgrade proceed as expected on the individual virtual machines, but the process of running the ARM template stopped for clients.

After you receive the timeout error, do not close Cloud Builder until you determine if Portal for ArcGIS is running on the virtual machine (or machines). To check this, complete the following steps:

1. Make a remote desktop connection to the machine where Portal for ArcGIS resides.

   You must make a remote desktop connection to the Jump Box and then make a remote desktop connection to a portal machine. The names of the portal machines are listed in the Manage Machines screen in Cloud Builder.

   If you are creating or upgrading a highly available portal, connect to the primary machine first. Sign in to the machine as a Windows Administrator.

2. Open Windows Task Manager and review the Details tab for the presence of the Setup.exe process. If the Setup.exe process is present, Portal for ArcGIS is still installing. When the Setup.exe process disappears, or if it was not present to begin with, proceed to the next step.
3. Open the portal home page in a web browser on the portal machine using the URL https://localhost:7443/arcgis/home.
4. What you will see on the portal home page depends on how much of the portal creation or upgrade was completed before the template timed out.
   • If the home page shows the option to sign in to the portal, do not sign in. Proceed to the next step (step 5).
   • If the home page shows any of the following options, do not click them. If the portal is highly available, repeat steps 1 through 3 on the standby machine. When that is finished or if there is only one machine in the portal, proceed to the next step.
     • Create a portal
     • Join an existing portal
     • Update license
     • Upgrade
   • If the sign in option or the options listed above are not present, open the Windows Services app (services.msc) to confirm that the Portal for ArcGIS service is started. If it is, then proceed to the next step. If it is not started, do not proceed to the next step. Instead, contact Esri technical support (for customers in the United States of America) or your local Esri distributor (for customers outside the United States).
5. Go back to Cloud Builder, click Back to return to the summary page, and click Finish on the summary page to complete the creation or upgrade process.

I receive the message ResourcePurchaseValidationFailed: User failed validation to purchase resources. Error message: 'Legal terms have not been accepted for this item on this subscription. To accept legal terms, please go to the Azure portal and configure programmatic deployment for the Marketplace item or create it there for the first time when I attempt to deploy an ArcGIS Server site or ArcGIS Enterprise site using Cloud Builder.

Microsoft Azure requires you to accept their license agreement before you can use images from the Azure Marketplace. Your Azure subscription administrator must perform a set of manual steps in the Azure portal and accept the Azure legal terms once for each unique image before you can deploy a site using Cloud Builder.

I receive an unhandled exception error in Cloud Builder when the Microsoft Azure session token expires.

This typically occurs if you leave Cloud Builder open and idle for many hours. Cloud Builder currently is not getting a new token when the existing one expires. Close and reopen Cloud Builder after each individual deployment operation to get a new session token from Microsoft Azure.

I am unable to sign out of Cloud Builder.

To sign out of Cloud Builder, for example, if you have more than one Microsoft Azure account and need to switch the account you're signed in with, do the following:

1. Sign out of all other Microsoft Azure clients on that machine. If you remain signed in to Azure from another client, the session token persists.
2. Close and reopen your web browser.
3. Clear the browser cache, and delete all cookies and history when you clear the cache.

I receive a 401 error when I click a button or link in Cloud Builder.

Using the operations in Cloud Builder accesses the Microsoft Azure REST API, which requires authentication of your Microsoft Azure account. In most cases, you receive a 401 error because your connection token timed out. Exit and sign in again and repeat the operation.

I receive a 403 error when I try to create or add to a deployment in Azure using Cloud Builder.

When you create or access resources in Azure, your Microsoft Azure account must have privileges to access to that resource. If you receive a 403 error, take note of the resource listed in the error and contact your account administrator to ensure you have privileges to access that type of resource.

I receive the error Unable to download deployment content when using Microsoft Az.Storage 6.0.0 Microsoft PowerShell module with artifacts generated by Cloud Builder 11.2 and earlier to create or upgrade a deployment in Microsoft Azure.

A change in the New-AzStorageContainerSASToken cmdlet starting with 6.0.0 removed the leading question mark (?) in the Azure Shared Access Signature (SAS) token. Automation artifacts generated from older Cloud Builder releases do not account for this change.

Open the Deploy-AzureResourceGroup.ps1 or Upgrade-AzureResourceGroup.ps1 automation artifact file in a text editor before you deploy, and replace the existing line (the first block shown below and noted as Old text) with the new text that follows. Save the changes to the file.

Old text

$OptionalParameters[$ArtifactsLocationSasTokenName] = ConvertTo-SecureString -AsPlainText -Force `
            (New-AzStorageContainerSASToken -Container $StorageContainerName -Context $StorageAccount.Context -Permission r -ExpiryTime (Get-Date).AddHours(4))

New text

$SASToken = (New-AzStorageContainerSASToken -Container $StorageContainerName -Context $StorageAccount.Context -Permission r -ExpiryTime (Get-Date).AddHours(4))
if(-not($SASToken.StartsWith('?'))){
    $SASToken = '?' + $SASToken
}

$OptionalParameters[$ArtifactsLocationSasTokenName] = ConvertTo-SecureString $SASToken -AsPlainText -Force

Site administration

I cannot make a remote desktop connection to the Microsoft Azure virtual machine.

Confirm you typed the correct login and password. You should sign in with the login and password you specified for the machine administrator when you deployed your site using ArcGIS Enterprise Cloud Builder for Microsoft Azure.

If you typed the login and password correctly but still cannot make a remote desktop connection, it is likely that your organization's firewall is blocking access. Contact your network administrator to allow access to Microsoft Azure virtual machines through the firewall.

Nota:

You cannot make a remote desktop connection to a Microsoft Azure SQL Database, Azure SQL Managed Instance, or Azure Database for PostgreSQL. To connect one of these databases from a remote machine, you must first configure the firewall on the database's virtual machine to allow remote connections and add the IP address of the remote client to the database machine.

Administrative access is enabled through the web proxy by default, but I don't want it enabled.

Do the following to disable administrative access through the web proxy URL:

1. Make a remote desktop connection to each of the web proxy instances. Sign in using the machine administrator credentials you specified in Cloud Builder when you deployed the site.
2. Open the c:\inetpub\wwwroot\web.config file in a text editor.
3. Remove admin or portaladmin tokens from the match URL rules.

   For example, remove admin from the match URL tag of the "RP-HTTP-Server" rule name text before removing

   <rule name="RP-HTTP-Server" stopProcessing="true">
                       <match url="^[^/]+/(manager|admin|rest|services|mobile|tokens|login|help)(/?)(.*)" />
   

   After you remove the admin token, the tag will look like the following:

   <rule name="RP-HTTP-Server" stopProcessing="true">
                       <match url="^[^/]+/(manager|rest|services|mobile|tokens|login|help)(/?)(.*)" />
   

4. Save changes to the web.config file and close it.

After installing an SSL (TLS) certificate, the URL property on existing web service items—including the utility services that are included with Portal for ArcGIS—is not updated. They still use the original *.cloudapp.azure.com URL.

To correct this, use the Portal REST API to update the URL property. The URL of the Portal REST endpoint is in the format https://<portal machine>.<region>.cloudapp.azure.com:7443/arcgis/sharing/rest. Sign in using the portal administrator credentials you specified in Cloud Builder when you deployed ArcGIS Enterprise.

I can't open the Microsoft Azure portal, Azure Marketplace, or Azure pricing calculator.

You may be using a browser or device that is not supported by Microsoft Azure. See Microsoft Azure documentation for supported browsers and devices for information.

I receive an invalid token error when I use the webgisdr tool to restore a version 2 (V2) deployment.

If you encounter issues using the webgisdr tool on a V2 deployment, see this Esri technical article for instructions.

Upgrades

I receive the following error after I upgrade ArcGIS Server: The connection property set was missing a required property or the property value was unrecognized. Connection was attempted with an older version of SQL Server client communications software that is not supported.

You must install a newer version of the Microsoft SQL Server ODBC driver on the machines in the ArcGIS Server site, because the ones you currently have are no longer supported. See Software required to connect to SQL Server for a list of supported ODBC drivers for each ArcGIS Server release.

Publishing

Publishing to my portal on Microsoft Azure fails, and I receive the message Publish exception 'Exception: no protocol: null/admin/services/exists'.

Occasionally, a race condition occurs between the start of the portal and ArcGIS Server services, which results in an invalid configuration.

To correct this, make a remote desktop connection to the portal machine and restart the Portal for ArcGIS service. Sign in using the machine administrator credentials you specified in Cloud Builder or Microsoft PowerShell Desired State Configuration (DSC) when you deployed ArcGIS Enterprise.

General

I and other portal members intermittently receive HTTP 502 errors when connecting to ArcGIS Enterprise portals on Microsoft Azure.

Beginning with 10.7.1, the load balancer probe port for Portal for ArcGIS must be configured to communicate over port 7443. If you upgraded from a 10.7 or earlier release portal, you must reconfigure the load balancer probe port, changing it from 7080 to 7443.

To change the port number, sign in to your account in the Microsoft Azure Portal, browse to the Azure internal load balancer associated with each of your Portal for ArcGIS virtual machines, and update the port value for the lbprobe. For further instructions on editing ports for internal load balancers, see the Microsoft Azure documentation.

Can I access data stored in Microsoft Azure from an ArcGIS client running on premises?

Though you can access databases running in the cloud or files stored in Azure Blob storage from ArcGIS software installed on premises, doing so will almost always be slower than accessing the data from a client installed in the same region in the cloud.

For this reason, Esri recommends you only access data in Azure from clients running in the same Azure region where the data is stored.